Text-to-image AI is mainstream now, but just waiting in the wings is text-to-video. The pitch for this technology is that you’ll be able to type a description and generate a corresponding video in any style you like.
Current capabilities lag behind this dream, but for those tracking the tech’s progress, an announcement today by AI startup Runway of a new AI video generation model is noteworthy nonetheless.
Runway offers a web-based video editor that specializes in AI tools like background removal and pose detection. The company helped develop open-source text-to-image model Stable Diffusion and announced its first AI video editing model, Gen-1, in February.
Gen-1 focused on transforming existing video footage, letting users input a rough 3D animation or shaky smartphone clip and apply an AI-generated overlay. In the clip below, for example, footage of cardboard packaging is paired with an image of an industrial factory to produce a clip that could be used for storyboarding or pitching a more polished feature.
