search

Picsart-AI-Research / StreamingT2V

StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text
https://streamingt2v.github.io/
1.1k stars 99 forks source link

Enhanced video length limited to 4sec #21

Open ajikmr opened 2 months ago

ajikmr commented 2 months ago

I am trying to generate 248 frame long sequence. It generates low resolution video of 31 seconds, but the final enhanced video is only 4 sec long. I found this for many different cases with varying frame sizes. Why is enhanced (high res) video length limited to 4sec?

Please find a sample case with output files here. https://drive.google.com/drive/folders/1eUNgtO_Ndf0qCza31BzgPEsU4Y75HZl8?usp=sharing

burnquiet commented 2 months ago

File: site-packages/modelscope/pipelines/multi_modal/video_to_video_pipeline.py In Line 63,it defines: max_frames = self.model.cfg.max_frames which is 32, i just modified it and it works.

oldnaari commented 2 months ago

Hello @ajikmr. Thank you for your report. Do you get this problem with inference.py or gradio_demo.py?

ajikmr commented 2 months ago

I used inference.py to run case. I will also look into fix suggested by @burnquiet Thanks.

PiPiNam commented 2 months ago

I also get this problem. And I wanana know should I modify the "max_frames = self.model.cfg.max_frames" to "max_frames = xxx (I set)" every time?

PiPiNam commented 2 months ago

I use A100 (80GB) to do inference with 600 frames.

However, the OOM error come out like below:

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 12.30 GiB (GPU 0; 79.15 GiB total capacity; 60.58 GiB already allocated; 10.64 GiB free; 67.98 GiB reserved in total by PyTorch) If reserved memory iement and PYTORCH_CUDA_ALLOC_CONF

So, I wonder how to produce the long enhanced video?

@oldnaari @burnquiet @ajikmr