Open kaijingxjtu opened 10 months ago
you can set :
But, I've seen few examples of successful sampling using Gpus with 24GB or smaller memory. Why is that?
They use smaller lower decoding frames. I would suggest you start with 14 and decrease it incrementally until you generate a video.
@zhanghongyong123456 this doesn't work for me...still OOM, can you share more about you GPU?
reduce decoding_t to 1, When you decode more than a dozen frames at the same time, it will explode
I believe ComfyUI is a better choice for GPU with memory less than 80G, as it use xformer
to reduce memory cost.
me too
@zhanghongyong123456 this doesn't work for me...still OOM, can you share more about you GPU?
I test on RTX 3090 (24G) and RTX 8000 (48G)
I saw someone uploaded svd-f16.safetentors version,though it is not offical,maybe you can try
And To set lowvram = True will really help,but just for scripts which import the streamlit_helpers.py, unfortunately SVD-series are not included
out of memory with 40G A100 GPU
I use default "Decode 14 frames at a time", but it sames OOM at the last step of sampling. But, I've seen few examples of successful sampling using Gpus with 24GB or smaller memory. Why is that?
`Collecting usage statistics. To deactivate, set browser.gatherUsageStats to False.
2023-11-24 13:24:33.363 Did not auto detect external IP. Please go to https://docs.streamlit.io/ for debugging hints.
You can now view your Streamlit app in your browser.
Network URL: http://xxx.xxx.xxx.xxx:80
No SDP backend available, likely because you are running in pytorch versions < 2.0. In fact, you are using PyTorch 1.13.1+cu117. You might want to consider upgrading. VideoTransformerBlock is using checkpointing VideoTransformerBlock is using checkpointing VideoTransformerBlock is using checkpointing VideoTransformerBlock is using checkpointing VideoTransformerBlock is using checkpointing VideoTransformerBlock is using checkpointing VideoTransformerBlock is using checkpointing VideoTransformerBlock is using checkpointing VideoTransformerBlock is using checkpointing VideoTransformerBlock is using checkpointing VideoTransformerBlock is using checkpointing VideoTransformerBlock is using checkpointing VideoTransformerBlock is using checkpointing VideoTransformerBlock is using checkpointing VideoTransformerBlock is using checkpointing VideoTransformerBlock is using checkpointing Initialized embedder #0: FrozenOpenCLIPImagePredictionEmbedder with 683800065 params. Trainable: False Initialized embedder #1: ConcatTimestepEmbedderND with 0 params. Trainable: False Initialized embedder #2: ConcatTimestepEmbedderND with 0 params. Trainable: False Initialized embedder #3: VideoPredictionEmbedderWithEncoder with 83653863 params. Trainable: False Initialized embedder #4: ConcatTimestepEmbedderND with 0 params. Trainable: False Loading model from /root/makaijing/generative-models-main/stable-video-diffusion-img2vid/svd.safetensors 2023-11-24 13:25:24.965 Uncaught app exception Traceback (most recent call last): File "/root/miniconda3/envs/vid_gen/lib/python3.8/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 534, in _run_script exec(code, module.dict) File "/root/makaijing/generative-models-main/scripts/demo/video_sampling.py", line 142, in
value_dict["cond_frames"] = img + cond_aug torch.randn_like(img)
TypeError: randn_like(): argument 'input' (position 1) must be Tensor, not NoneType
Global seed set to 23
Global seed set to 23
Global seed set to 23
############################## Sampling setting ##############################
Sampler: EulerEDMSampler
Discretization: EDMDiscretization
Guider: LinearPredictionGuider
Sampling with EulerEDMSampler for 26 steps: 0%| | 0/26 [00:00<?, ?it/s]/root/miniconda3/envs/vid_gen/lib/python3.8/site-packages/torch/utils/checkpoint.py:31: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
warnings.warn("None of the inputs have requires_grad=True. Gradients will be None")
Sampling with EulerEDMSampler for 26 steps: 96%|████████████████████████████████████████████████████▉ | 25/26 [01:02<00:02, 2.48s/it]
2023-11-24 13:27:05.104 Uncaught app exception
Traceback (most recent call last):
File "/root/miniconda3/envs/vid_gen/lib/python3.8/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 534, in _run_script
exec(code, module.dict)
File "/root/makaijing/generative-models-main/scripts/demo/video_sampling.py", line 174, in
out = do_sample(
File "/root/makaijing/generative-models-main/./scripts/demo/streamlit_helpers.py", line 616, in do_sample
samples_x = model.decode_first_stage(samples_z)
File "/root/miniconda3/envs/vid_gen/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func( args, kwargs)
File "/root/makaijing/generative-models-main/./sgm/models/diffusion.py", line 130, in decode_first_stage
out = self.first_stage_model.decode(
File "/root/makaijing/generative-models-main/./sgm/models/autoencoder.py", line 211, in decode
x = self.decoder(z, kwargs)
File "/root/miniconda3/envs/vid_gen/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(input, kwargs)
File "/root/makaijing/generative-models-main/./sgm/modules/diffusionmodules/model.py", line 733, in forward
h = self.up[i_level].block[i_block](h, temb, kwargs)
File "/root/miniconda3/envs/vid_gen/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(input, *kwargs)
File "/root/makaijing/generative-models-main/./sgm/modules/autoencoding/temporal_ae.py", line 70, in forward
x = super().forward(x, temb)
File "/root/makaijing/generative-models-main/./sgm/modules/diffusionmodules/model.py", line 134, in forward
h = nonlinearity(h)
File "/root/makaijing/generative-models-main/./sgm/modules/diffusionmodules/model.py", line 49, in nonlinearity
return x torch.sigmoid(x)
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 3.94 GiB (GPU 0; 31.75 GiB total capacity; 23.74 GiB already allocated; 3.19 GiB free; 27.38 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF`
ffmpeg version 3.4.8-0ubuntu0.2 Copyright (c) 2000-2020 the FFmpeg developers built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04) configuration: --prefix=/usr --extra-version=0ubuntu0.2 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared libavutil 55. 78.100 / 55. 78.100 libavcodec 57.107.100 / 57.107.100 libavformat 57. 83.100 / 57. 83.100 libavdevice 57. 10.100 / 57. 10.100 libavfilter 6.107.100 / 6.107.100 libavresample 3. 7. 0 / 3. 7. 0 libswscale 4. 8.100 / 4. 8.100 libswresample 2. 9.100 / 2. 9.100 libpostproc 54. 7.100 / 54. 7.100