How to generate good videos?

[jzhang38]

Hi, thanks for dropping a wonderful model!

I cannot generate videos matching the demo quality using the command given in the readme, even with the prompt given in the demos:

python inference.py --ckpt_dir LTX-Video- --prompt "The turquoise waves crash against the dark, jagged rocks of the shore, sending white foam spraying into the air. The scene is dominated by the stark contrast between the bright blue water and the dark, almost black rocks. The water is a clear, turquoise color, and the waves are capped with white foam. The rocks are dark and jagged, and they are covered in patches of green moss. The shore is lined with lush green vegetation, including trees and bushes. In the background, there are rolling hills covered in dense forest. The sky is cloudy, and the light is dim." --height 720 --width 1280 --num_frames 257 --seed 42

I used 720 x 1280, 257 frames and seed 42.

Is there anything I can do to increase the quality? Did I do anything wrong? I want to ask if the videos given in the demos are generated with the default steps (40) or more?

[jzhang38]

Some samples of videos generated by me:

https://github.com/user-attachments/assets/82069dee-dd6e-4599-8595-b4bff755a28d

The turquoise waves crash against the dark, jagged rocks of the shore, sending white foam spraying into the air. The scene is dominated by the stark contrast between the bright blue water and the dark, almost black rocks. The water is a clear, turquoise color, and the waves are capped with white foam. The rocks are dark and jagged, and they are covered in patches of green moss. The shore is lined with lush green vegetation, including trees and bushes. In the background, there are rolling hills covered in dense forest. The sky is cloudy, and the light is dim.

https://github.com/user-attachments/assets/b6743d24-016a-4143-a255-24dd2b9e0dae

A narrow dirt trail winds through a meadow blanketed in wildflowers, their vibrant yellows and purples swaying gently in the breeze. The air hums with the soft buzz of bees and the occasional chirp of a bird. In the distance, a lone deer grazes near a cluster of oak trees, its ears flicking at the slightest sound. Above, clouds drift lazily across a pale blue sky, casting fleeting shadows over the rolling hills.

https://github.com/user-attachments/assets/a9a2438b-1d1d-413a-b3e9-6721780de1d9

A weathered barn stands at the edge of a sprawling field, its red paint faded to a rustic charm. Tall stalks of corn ripple in waves as the wind sweeps through, their golden tassels glinting in the sunlight. A tractor idles near the barn, its tires caked in dry soil. In the background, a row of cottonwood trees lines a gravel driveway, their leaves rustling like whispers in the warm afternoon air.

https://github.com/user-attachments/assets/ab708fe3-53e7-431e-b5eb-1f6abbace687

A young woman sits on a weathered wooden bench in the park, her head tilted down as she reads a worn paperback. Her loose-knit sweater drapes softly over her shoulders, catching the golden light of the setting sun. Nearby, leaves rustle gently in the breeze, scattering shades of amber and crimson across the gravel path. Her expression is one of quiet concentration, her surroundings a bubble of tranquil isolation amidst the distant hum of city life.

There is barely any motion

[cchance27]

"The model works best on resolutions under 720 x 1280 and number of frames below 257."

257, isn't below 257, i'd follow a much smaller generation sample, also from what i've seen the 512x768 is the standard resolution they say they support 720x1280 but all the good videos i've seen have been 512x768

[jzhang38]

Some samples with 512 X 768 X 129 as suggested by @cchance27 . I think it is much better now:

https://github.com/user-attachments/assets/32884487-9981-447b-be37-9f021d1c743f

A weathered barn stands at the edge of a sprawling field, its red paint faded to a rustic charm. Tall stalks of corn ripple in waves as the wind sweeps through, their golden tassels glinting in the sunlight. A tractor idles near the barn, its tires caked in dry soil. In the background, a row of cottonwood trees lines a gravel driveway, their leaves rustling like whispers in the warm afternoon air.

https://github.com/user-attachments/assets/88246789-adc9-4bc7-9c04-6a8abc04870e

A lone sailboat drifts lazily across the horizon, its white sails billowing softly in the sea breeze. The water below sparkles with flecks of silver, the gentle waves rolling against the shore with a rhythmic sigh. On the beach, smooth pebbles glisten under the sun, scattered among driftwood and clusters of seaweed. In the distance, cliffs rise majestically, their rocky faces streaked with trails of cascading greenery.

https://github.com/user-attachments/assets/e37e5a1c-e049-4abf-83e3-5e92c54afe46

A vintage train snakes through the mountains, its plume of white steam rising dramatically against the jagged peaks. The cars glint in the late afternoon sun, their deep crimson and gold accents lending a touch of elegance. The tracks carve a precarious path along the cliffside, revealing glimpses of a roaring river far below. Inside, passengers peer out the large windows, their faces lit with awe as the landscape unfolds.

A crowded rooftop bar buzzes with energy, the city skyline twinkling like a field of stars in the background. Strings of fairy lights hang above, casting a warm, golden glow over the scene. Groups of people gather around high tables, their laughter blending with the soft rhythm of live jazz. The aroma of freshly mixed cocktails and charred appetizers wafts through the air, mingling with the cool night breeze

https://github.com/user-attachments/assets/482a919c-52ff-483a-92e9-27a0aed8a0e0

A crowded rooftop bar buzzes with energy, the city skyline twinkling like a field of stars in the background. Strings of fairy lights hang above, casting a warm, golden glow over the scene. Groups of people gather around high tables, their laughter blending with the soft rhythm of live jazz. The aroma of freshly mixed cocktails and charred appetizers wafts through the air, mingling with the cool night breeze.

[dshalem]

Hi @jzhang38, This is a result of High number of frames on high resolution. We are aware that increasing the frames can cause the video to look blurry and we are working on it. I see that your first videos are 720 x 1280, 257 frames, as you mentioned, which can indeed cause this issue. If you are using comfy you can enable the "sigma stretch" terminal to 0.1 as it will help with High number of frames on high resolution. We are working on it now to enable it in the best way also in inference code.

You can also use the presets for now. For example, 121 frames on 512 X 768.