Image to Video

Animate a still image into a video clip in PHP with local Stable Video Diffusion, Wan2.1 or CogVideoX image-to-video models.

Animate a still image into a short video clip with a local diffusers image-to-video model, optionally guided by a prompt.

Setup#

Terminal

vendor/bin/loves-ai setup image-to-video
vendor/bin/loves-ai pull stabilityai/stable-video-diffusion-img2vid-xt

Supported models#

Models differ in what they take:

Image only, e.g. stabilityai/stable-video-diffusion-img2vid-xt. It animates the picture on its own and refuses a prompt.
Image and prompt, e.g. Wan-AI/Wan2.1-I2V-14B-480P-Diffusers, zai-org/CogVideoX-5b-I2V or LTX-Video. The prompt says what should happen in the clip.

A model whose repository only ships its text-to-video pipeline is converted to the image-to-video pipeline of the same family automatically, when diffusers has one.

As with text to video, these models are large and slow, so always generate in a queue job.

From the command line#

Terminal

vendor/bin/loves-ai image-to-video stabilityai/stable-video-diffusion-img2vid-xt photo.jpg --output=clip.mp4

Output

🎬 Bringing your picture to life with stabilityai/stable-video-diffusion-img2vid-xt… Films take their time — perfect for a pot of tea and some cookies 🍪
If you wish to see all logs, re-run the command with the "--debug" option.
🎉 Video saved to /var/www/my-app/clip.mp4

Options#

The options are the same as for text to video, plus one:

Option	Meaning
`--prompt=TEXT`	What should happen in the clip, for models that accept a prompt

Defaults come from config/image-to-video.php (output_dir, log_file).

From PHP#

PHP

use PhpLovesAi\Runner\ImageToVideo;

// Finds the runner and the pulled model in the project's .local directory by itself.
$clip = (new ImageToVideo())->generate(
    model: 'Wan-AI/Wan2.1-I2V-14B-480P-Diffusers',
    imagePath: storage_path('app/photo.jpg'),
    outputPath: storage_path('app/clip.mp4'),
    prompt: 'the camera slowly zooms out',
    frames: 33,
    fps: 16,
);

Errors#

ImageNotFoundException when the image does not exist
BinaryNotInstalledException when setup image-to-video has not been run
ModelNotFoundException when the model was not pulled yet
UnsupportedModelException when the model is not a diffusers video pipeline
RunFailedException when the run fails, e.g. when the model takes no prompt but one was given