GARV-k
commented
1 month ago Same here. Would request the repo maintainers to comment about it.!
Open zephirusgit opened 1 year ago
GARV-k
commented
1 month ago Same here. Would request the repo maintainers to comment about it.!
johndpope
commented
1 month ago
Could someone give me a hand, I have not been able to get this working, I am using a 12gb rtx2060, I am using it with miniconda3, for the env, in windows, and I tried to find out what the error is but I did not find anything that is clear enough, yes It is the reserved memory, I don't know how to change that, I share the nvidia-smi info the demo.py that I put together, pip list, and what it tells me.
how to install it conda create -n textovideo python=3.9 pip -y conda activate textovideo pip install -r requirements.txt pip3 install numpy --pre torch torchvision torchaudio --force-reinstall --index-url https://download.pytorch.org/whl/nightly/cu118
pip install kwargs¿?¿?? pip install ffprobe?¿?
(demo.py) .................................................................... import torch from model import Model model = Model(device = "cuda", dtype = torch.float16)
prompt = 'oil painting of a deer, a high-quality, detailed, and professional photo' video_path = 'assets/depth_videos/deer.mp4' out_path = f'./text2video_depthcontrol{prompt}.mp4' model.process_controlnet_depth(video_path, prompt=prompt, save_path=out_path) .................................................................... nvidia-smi Mon May 8 06:18:38 2023 +---------------------------------------------------------------------------------------+ | NVIDIA-SMI 531.14 Driver Version: 531.14 CUDA Version: 12.1 | |-----------------------------------------+----------------------+----------------------+ | GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+======================+======================| | 0 NVIDIA GeForce RTX 2060 WDDM | 00000000:07:00.0 On | N/A | | 38% 50C P0 36W / 184W| 381MiB / 12288MiB | 0% Default | | | | N/A | +-----------------------------------------+----------------------+----------------------+
absl-py 1.4.0 accelerate 0.16.0 addict 2.4.0 aiofiles 23.1.0 aiohttp 3.8.4 aiosignal 1.3.1 albumentations 1.3.0 altair 4.2.2 antlr4-python3-runtime 4.9.3 anyio 3.6.2 args 0.1.0 async-timeout 4.0.2 attrs 23.1.0 basicsr 1.4.2 beautifulsoup4 4.12.2 braceexpand 0.1.7 bs4 0.0.1 cachetools 5.3.0 certifi 2022.12.7 charset-normalizer 2.1.1 click 8.1.3 colorama 0.4.6 coloredlogs 15.0.1 contourpy 1.0.7 cycler 0.11.0 decorator 4.4.2 decord 0.6.0 diffusers 0.14.0 einops 0.6.0 entrypoints 0.4 fastapi 0.95.1 ffmpy 0.3.0 ffprobe 0.5 filelock 3.9.0 flatbuffers 23.3.3 fonttools 4.39.3 frozenlist 1.3.3 fsspec 2023.4.0 ftfy 6.1.1 future 0.18.3 google-auth 2.17.3 google-auth-oauthlib 1.0.0 gradio 3.23.0 grpcio 1.54.0 h11 0.14.0 httpcore 0.17.0 httpx 0.24.0 huggingface-hub 0.14.1 humanfriendly 10.0 idna 3.4 imageio 2.9.0 imageio-ffmpeg 0.4.2 importlib-metadata 6.6.0 importlib-resources 5.12.0 invisible-watermark 0.1.5 Jinja2 3.1.2 joblib 1.2.0 jsonschema 4.17.3 kiwisolver 1.4.4 kornia 0.6.0 kwargs 1.0.1 linkify-it-py 2.0.2 lmdb 1.4.1 Markdown 3.4.3 markdown-it-py 2.2.0 MarkupSafe 2.1.2 matplotlib 3.7.1 mdit-py-plugins 0.3.3 mdurl 0.1.2 moviepy 1.0.3 mpmath 1.2.1 multidict 6.0.4 networkx 3.0rc1 numpy 1.24.1 oauthlib 3.2.2 omegaconf 2.3.0 onnx 1.14.0 onnxruntime 1.14.1 open-clip-torch 2.16.0 opencv-contrib-python 4.7.0.72 opencv-python 4.7.0.72 opencv-python-headless 4.7.0.72 orjson 3.8.11 packaging 23.1 pandas 2.0.1 Pillow 9.3.0 pip 23.0.1 prettytable 3.6.0 proglog 0.1.10 protobuf 3.20.3 psutil 5.9.5 pyasn1 0.5.0 pyasn1-modules 0.3.0 pydantic 1.10.7 pyDeprecate 0.3.1 pydub 0.25.1 pyparsing 3.0.9 pyreadline3 3.4.1 pyrsistent 0.19.3 python-dateutil 2.8.2 python-multipart 0.0.6 pytorch-lightning 1.5.0 pytz 2023.3 PyWavelets 1.4.1 PyYAML 6.0 qudida 0.0.4 regex 2023.5.5 requests 2.28.1 requests-oauthlib 1.3.1 rsa 4.9 safetensors 0.2.7 scikit-image 0.19.3 scikit-learn 1.2.2 scipy 1.10.1 semantic-version 2.10.0 sentencepiece 0.1.99 setuptools 66.0.0 six 1.16.0 sniffio 1.3.0 soupsieve 2.4.1 starlette 0.26.1 sympy 1.11.1 tb-nightly 2.14.0a20230506 tensorboard 2.13.0 tensorboard-data-server 0.7.0 tensorboardX 2.6 test-tube 0.7.5 threadpoolctl 3.1.0 tifffile 2023.4.12 timm 0.6.12 tokenizers 0.13.3 tomesd 0.1.2 toolz 0.12.0 torch 2.1.0.dev20230506+cu118 torchaudio 2.1.0.dev20230507+cu118 torchmetrics 0.6.0 torchvision 0.16.0.dev20230507+cu118 tqdm 4.64.1 transformers 4.26.0 typing_extensions 4.4.0 tzdata 2023.3 uc-micro-py 1.0.2 urllib3 1.26.13 uvicorn 0.22.0 wcwidth 0.2.6 webdataset 0.2.5 websockets 11.0.2 Werkzeug 2.3.3 wheel 0.38.4 yapf 0.32.0 yarl 1.9.2 zipp 3.15.0
the errors
(textovideo) PS H:\ia\Text2Video-Zero-main> python demo.py H:\ia\Text2Video-Zero-main\annotator\openpose\body.py:5: DeprecationWarning: Please use
model.process_controlnet_depth(video_path, prompt=prompt, save_path=out_path)
File "H:\ia\Text2Video-Zero-main\model.py", line 243, in process_controlnet_depth
result = self.inference(image=control,
File "H:\ia\Text2Video-Zero-main\model.py", line 120, in inference
result.append(self.inference_chunk(frame_ids=frame_ids,
File "H:\ia\Text2Video-Zero-main\model.py", line 79, in inference_chunk
return self.pipe(prompt=prompt[frame_ids].tolist(),
File "C:\Users\ultim\miniconda3\envs\textovideo\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, kwargs)
File "C:\Users\ultim\miniconda3\envs\textovideo\lib\site-packages\diffusers\pipelines\stable_diffusion\pipeline_stable_diffusion_controlnet.py", line 749, in call
down_block_res_samples, mid_block_res_sample = self.controlnet(
File "C:\Users\ultim\miniconda3\envs\textovideo\lib\site-packages\torch\nn\modules\module.py", line 1502, in _wrapped_call_impl
return self._call_impl(*args, *kwargs)
File "C:\Users\ultim\miniconda3\envs\textovideo\lib\site-packages\torch\nn\modules\module.py", line 1511, in _call_impl
return forward_call(args, kwargs)
File "C:\Users\ultim\miniconda3\envs\textovideo\lib\site-packages\diffusers\models\controlnet.py", line 461, in forward
sample, res_samples = downsample_block(
File "C:\Users\ultim\miniconda3\envs\textovideo\lib\site-packages\torch\nn\modules\module.py", line 1502, in _wrapped_call_impl
return self._call_impl(*args, kwargs)
File "C:\Users\ultim\miniconda3\envs\textovideo\lib\site-packages\torch\nn\modules\module.py", line 1511, in _call_impl
return forward_call(*args, *kwargs)
File "C:\Users\ultim\miniconda3\envs\textovideo\lib\site-packages\diffusers\models\unet_2d_blocks.py", line 837, in forward
hidden_states = attn(
File "C:\Users\ultim\miniconda3\envs\textovideo\lib\site-packages\torch\nn\modules\module.py", line 1502, in _wrapped_call_impl
return self._call_impl(args, kwargs)
File "C:\Users\ultim\miniconda3\envs\textovideo\lib\site-packages\torch\nn\modules\module.py", line 1511, in _call_impl
return forward_call(*args, kwargs)
File "C:\Users\ultim\miniconda3\envs\textovideo\lib\site-packages\diffusers\models\transformer_2d.py", line 265, in forward
hidden_states = block(
File "C:\Users\ultim\miniconda3\envs\textovideo\lib\site-packages\torch\nn\modules\module.py", line 1502, in _wrapped_call_impl
return self._call_impl(*args, *kwargs)
File "C:\Users\ultim\miniconda3\envs\textovideo\lib\site-packages\torch\nn\modules\module.py", line 1511, in _call_impl
return forward_call(args, kwargs)
File "C:\Users\ultim\miniconda3\envs\textovideo\lib\site-packages\diffusers\models\attention.py", line 291, in forward
attn_output = self.attn1(
File "C:\Users\ultim\miniconda3\envs\textovideo\lib\site-packages\torch\nn\modules\module.py", line 1502, in _wrapped_call_impl
return self._call_impl(*args, *kwargs)
File "C:\Users\ultim\miniconda3\envs\textovideo\lib\site-packages\torch\nn\modules\module.py", line 1511, in _call_impl
return forward_call(args, **kwargs)
File "C:\Users\ultim\miniconda3\envs\textovideo\lib\site-packages\diffusers\models\cross_attention.py", line 205, in forward
return self.processor(
File "H:\ia\Text2Video-Zero-main\utils.py", line 218, in call
attention_probs = attn.get_attention_scores(query, key, attention_mask)
File "C:\Users\ultim\miniconda3\envs\textovideo\lib\site-packages\diffusers\models\cross_attention.py", line 242, in get_attention_scores
attention_scores = torch.baddbmm(
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 4.00 GiB. GPU 0 has a total capacty of 12.00 GiB of which 555.75 MiB is free. Of the allocated memory 7.91 GiB is allocated by PyTorch, and 658.32 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
(textovideo) PS H:\ia\Text2Video-Zero-main>
gaussian_filterfrom thescipy.ndimagenamespace, thescipy.ndimage.filtersnamespace is deprecated. from scipy.ndimage.filters import gaussian_filter H:\ia\Text2Video-Zero-main\annotator\openpose\hand.py:6: DeprecationWarning: Please usegaussian_filterfrom thescipy.ndimagenamespace, thescipy.ndimage.filtersnamespace is deprecated. from scipy.ndimage.filters import gaussianfilter C:\Users\ultim\miniconda3\envs\textovideo\lib\site-packages\skimage\util\dtype.py:27: DeprecationWarning:np.bool8is a deprecated alias for `np.bool. (Deprecated NumPy 1.24) np.bool8: (False, True), cuda cuda Module Depth C:\Users\ultim\miniconda3\envs\textovideo\lib\site-packages\safetensors\torch.py:98: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage() with safe_open(filename, framework="pt", device=device) as f: text_encoder\model.safetensors not found Fetching 15 files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████| 15/15 [00:00<?, ?it/s] You have disabled the safety checker for <class 'diffusers.pipelines.stable_diffusion.pipeline_stable_diffusion_controlnet.StableDiffusionControlNetPipeline'> by passingsafety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 . Processing chunk 1 / 2 0%| | 0/20 [00:01<?, ?it/s] Traceback (most recent call last): File "H:\ia\Text2Video-Zero-main\demo.py", line 8, inThank you very much in advance if you can answer and guide me a bit, with other things I have not had problems, eg stable diff, stable diff fast, when I ran the demos I downloaded a number of things,
07/05/2023 06:51 209.267.595 body_pose_model.pth 07/05/2023 06:32 13 ckpts.txt 07/05/2023 07:01 492.757.791 dpt_hybrid-midas-501f0c75.pt 07/05/2023 06:53 147.341.049 hand_pose_model.pth