Google Launches New Video-Generating AI Model Called Lumiere

Google เปิดตัวโมเดล AI สร้างวิดีโอตัวใหม่ ชื่อ Lumiere

It is an AI model that creates videos at the state-of-the-art level called Lumiere.

Originally, Google had a video maker called VideoPoet already

VideoPoet – Google Research
A Large Language Model for Zero-Shot Video Generation. VideoPoet demonstrates simple modeling method that can convert any autoregressive language model into a high quality video generator.

with the name shape generator. ImageGen  

Image Video
High-definition video Generation with Diffusion Models

But Lumiere came out. This is much cooler than the original, and it can create high-quality short-form videos.

Basics

  • Lumiere is trained with a dataset of over 30 million videos. 80 frames long with text captions inserted to practice it.
  • It can produce 16 frames of video per second.
  • It is not said that the 30 million videos are taken from the same source.

This work is a joint research between the Google Research team and Israel's leading technology universities, namely:

Lumiere has 4 cool features:

  1. Video Editing - You can solve it by prompting the command (easy but very divine).
  1. Stylized Generation - You can create a video from the reference we provide (you can give the reference as an image or video). The model will learn the style and then create a video to use that style.
  1. Image & Text to Video - Don't just put a prompt as a message. For example, throw a picture of us with a still face and then tell it to create a picture of us laughing. It can be done, it's too godly.
  1. Cinemagraphs - Select only a part of the image and animate it into a video.

source

All of this information is Google research published on GitHub.

Lumiere - Google ResearchSpace-Time Text-to-Video diffusion model by Google Research.lumiere-video.github.io

Paper งานวิจัย

https://arxiv.org/abs/2401.12945
https://arxiv.org/abs/2401.12945

One of the researchers, named Hila Chefs From Tel Aviv University, it was posted on X.

Recommend other interesting websites and newsletters.

Great! Next, complete checkout for full access to The Insiderly AI.
Welcome back! You've successfully signed in.
You've successfully subscribed to The Insiderly AI.
Success! Your account is fully activated, you now have access to all content.
Success! Your billing info has been updated.
Your billing was not updated.