Nvidia’s text-to-video tech will take your GIF game to the next level

Two laptops showing gifs generated by Nvidia

Now that ChatGPT and Midjourney are just about mainstream, the following massive AI race is text-to-video turbines – and Nvidia has simply proven off some spectacular demos of the tech that would quickly take your GIFs to a brand new stage.

A new research paper and micro-site (opens in new tab) from Nvidia’s Toronto AI Lab, known as “Excessive-Decision Video Synthesis with Latent Diffusion Fashions”, provides us a style of the unimaginable video creation instruments which can be about to hitch the ever-growing checklist of one of the best AI artwork turbines.

Latent Diffusion Fashions (or LDMs) are a sort of AI that may generate movies without having large computing energy. Nvidia says its tech does this by constructing on the work of text-to-image turbines, on this case Steady Diffusion, and including a “temporal dimension to the latent area diffusion mannequin”.

(Picture credit score: Nvidia)

In different phrases, its generative AI could make nonetheless photos transfer in a sensible manner and upscale them to utilizing super-resolution strategies. This implies it will possibly produce quick, 4.7-second lengthy movies with a decision of 1280×2048, or longer ones on the decrease decision of 512×1024 for driving movies.

Our rapid thought on seeing the early demos (like those above and beneath) is how a lot this might increase our GIF sport. Okay, there are greater ramifications, just like the democratization of video creation and the prospect of automated movie variations, however at this stage text-to-GIF appears to be probably the most thrilling use case.

A teddy bear playing the electric guitar

(Picture credit score: Nvidia)

Easy prompts like ‘a storm trooper vacuuming on the seaside’ and a ‘teddy bear is taking part in the electrical guitar, excessive definition, 4K’ produce some fairly usable outcomes, even when there are of course artifacts and morphing with a few of the creations.