RuntimeError: Error(s) in loading state_dict for StreamingSVD: Unexpected key(s) in state_dict: "conditioner.embedders.0.open_clip.model.ln_final.bias"

[jiyuwangbupt]

--- START: Loading module: conditioner --- Warning: Seems object conditioner was not build correct. --- FINSHED: Loading module: conditioner ---

LOADING TRAINED WEIGHTS Traceback (most recent call last): File "/mnt/host/data/StreamingT2V/code/inference_i2v.py", line 218, in generator = StreamingPipeline() File "/mnt/host/data/StreamingT2V/code/inference_i2v.py", line 62, in init return_dict = self.init_model(cli) File "/mnt/host/data/StreamingT2V/code/inference_i2v.py", line 138, in init_model model.load_state_dict(ckpt) # load trained model File "/data/eas/ENV/lib/python3.9/site-packages/torch/nn/modules/module.py", line 2041, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for StreamingSVD: Unexpected key(s) in state_dict: "conditioner.embedders.0.open_clip.model.ln_final.bias", "conditioner.embedders.0.open_clip.model.ln_final.weight"

[rob-hen]

Hi @jiyuwangbupt ,

please check this issue: https://github.com/Picsart-AI-Research/StreamingT2V/issues/59

[jiyuwangbupt]

Hi @jiyuwangbupt ,

please check this issue: #59

I created an image and ran it in a Docker container.

pip freeze >> requirements.txt

torch==2.0.1 torchvision==0.15.2 diffusers==0.28.0 imageio==2.34.0 imageio-ffmpeg==0.4.9 transformers==4.40.0 numpy==1.26.4 einops==0.7.0 pillow==10.3.0 timm==1.0.7 tokenizers==0.19.1 triton==2.0.0 typing_extensions==4.11.0 torchsde==0.2.6 tqdm==4.66.2 pytorch-lightning==2.2.2 jsonargparse==4.28.0 matplotlib==3.8.4 omegaconf==2.3.0 typeshed_client==2.5.1 docstring_parser==0.16 kornia==0.7.2 open-clip-torch==2.24.0 xformers==0.0.20 opencv-python==4.10.0.84 accelerate==0.29.3 gdown==5.2.0absl-py==2.1.0 accelerate==0.29.3 aiohappyeyeballs==2.4.2 aiohttp==3.10.6 aiosignal==1.3.1 aliyun-python-sdk-core==2.13.13 aliyun-python-sdk-core-v3==2.13.11 aliyun-python-sdk-kms==2.8.0 annotated-types==0.7.0 antlr4-python3-runtime==4.9.3 anyio==4.4.0 async-timeout==4.0.3 attrs==24.2.0 av==12.3.0 bcrypt==4.2.0 beautifulsoup4==4.12.3 bitsandbytes==0.43.3 certifi==2024.7.4 cffi==1.17.1 cfgv==3.4.0 chardet==3.0.4 click==8.1.7 cmake==3.30.3 coloredlogs==15.0.1 colossalai==0.4.0 contexttimer==0.3.3 contourpy==1.3.0 crcmod==1.7 cryptography==43.0.1 cycler==0.12.1 decorator==5.1.1 decord==0.6.0 Deprecated==1.2.14 diffusers==0.28.0 distlib==0.3.8 docstring_parser==0.16 einops==0.7.0 exceptiongroup==1.2.2 fabric==3.2.2 fastapi==0.114.1 fig @ http://youkudnnmgv1.oss-cn-zhangjiakou.aliyuncs.com/USERS/ljt/dist/fig-0.0.0-py3-none-any.whl#sha256=3d14a42e3d8403f3816faf5dba3b9e991fcba6beb18bb4dd31eb28fe9c9547c7 filelock==3.15.4 flatbuffers==24.3.25 fonttools==4.53.1 frozenlist==1.4.1 fsspec==2024.6.1 ftfy==6.2.3 future==1.0.0 galore-torch==1.0 gdown==5.2.0 google==3.0.0 grpcio==1.66.1 h11==0.14.0 huggingface-hub==0.24.6 humanfriendly==10.0 identify==2.6.0 idna==2.10 imageio==2.34.0 imageio-ffmpeg==0.4.9 importlib_metadata==8.4.0 importlib_resources==6.4.4 invoke==2.2.0 Jinja2==3.1.4 jmespath==0.10.0 jsonargparse==4.28.0 jsonschema==4.23.0 jsonschema-specifications==2023.12.1 kiwisolver==1.4.5 kornia==0.7.2 kornia_rs==0.1.5 lazy_loader==0.4 lightning-utilities==0.11.7 lit==18.1.8 ljtcore @ http://youkudnnmgv1.oss-cn-zhangjiakou.aliyuncs.com/USERS/ljt/dist/ljtcore-1.0.3-py3-none-any.whl#sha256=38df10f247e0e21b210490df94a61fd9ecba0d0ab41480f8523ad3a934f44611 Markdown==3.7 markdown-it-py==3.0.0 MarkupSafe==2.1.5 matplotlib==3.8.4 mdurl==0.1.2 mpmath==1.3.0 msgpack==1.1.0 multidict==6.1.0 mypy-extensions==1.0.0 networkx==3.2.1 ninja==1.11.1.1 nodeenv==1.9.1 numpy==1.26.4 nvidia-cublas-cu11==11.10.3.66 nvidia-cublas-cu12==12.1.3.1 nvidia-cuda-cupti-cu11==11.7.101 nvidia-cuda-cupti-cu12==12.1.105 nvidia-cuda-nvrtc-cu11==11.7.99 nvidia-cuda-nvrtc-cu12==12.1.105 nvidia-cuda-runtime-cu11==11.7.99 nvidia-cuda-runtime-cu12==12.1.105 nvidia-cudnn-cu11==8.5.0.96 nvidia-cudnn-cu12==8.9.2.26 nvidia-cufft-cu11==10.9.0.58 nvidia-cufft-cu12==11.0.2.54 nvidia-curand-cu11==10.2.10.91 nvidia-curand-cu12==10.3.2.106 nvidia-cusolver-cu11==11.4.0.1 nvidia-cusolver-cu12==11.4.5.107 nvidia-cusparse-cu11==11.7.4.91 nvidia-cusparse-cu12==12.1.0.106 nvidia-nccl-cu11==2.14.3 nvidia-nccl-cu12==2.19.3 nvidia-nvjitlink-cu12==12.6.68 nvidia-nvtx-cu11==11.7.91 nvidia-nvtx-cu12==12.1.105 omegaconf==2.3.0 onnxruntime==1.16.3 open-clip-torch==2.24.0 opencv-python==4.10.0.84 oss2==2.5.0 packaging==24.1 pandas==2.2.2 paramiko==3.4.1 peft==0.12.0 pillow==10.3.0 platformdirs==4.3.2 plumbum==1.8.3 pre-commit==3.8.0 protobuf==5.28.0 psutil==6.0.0 pycparser==2.22 pycryptodome==3.20.0 pydantic==2.9.1 pydantic_core==2.23.3 Pygments==2.18.0 PyNaCl==1.5.0 pyparsing==3.1.4 pyre-extensions==0.0.29 PySocks==1.7.1 python-dateutil==2.9.0.post0 pytorch-lightning==2.2.2 pytz==2024.1 PyYAML==6.0.2 ray==2.35.0 referencing==0.35.1 regex==2024.7.24 requests==2.23.0 rich==13.8.1 rpds-py==0.20.0 rpyc==6.0.0 safetensors==0.4.4 scikit-image==0.24.0 scipy==1.13.1 sentencepiece==0.2.0 six==1.16.0 sniffio==1.3.1 soupsieve==2.6 starlette==0.38.5 sympy==1.13.2 tdqm==0.0.1 tensorboard==2.17.1 tensorboard-data-server==0.7.2 tifffile==2024.8.28 timm==1.0.7 tokenizers==0.19.1 torch==2.0.1 torchaudio==0.9.1 torchmetrics==1.4.2 torchsde==0.2.6 torchvision==0.15.2 tornado==5.1.1 tqdm==4.66.2 trampoline==0.1.2 transformers==4.40.0 triton==2.0.0 typeshed_client==2.5.1 typing-inspect==0.9.0 typing_extensions==4.11.0 tzdata==2024.1 urllib3==1.23 uvicorn==0.29.0

Editable install with no version control (videosys==2.0.0) -e /mnt/host/data/VideoSys virtualenv==20.26.4 wcwidth==0.2.13 Werkzeug==3.0.4 wrapt==1.16.0 xformers==0.0.20 yarl==1.13.0 yk-smartsdk==2.0.50 zipp==3.20.1

[jiyuwangbupt]

Hi @jiyuwangbupt ,

please check this issue: #59

root@a2f93364212e:/mnt/host/data/StreamingT2V# pip install -r requirements.txt Requirement already satisfied: torch==2.0.1 in /data/eas/ENV/lib/python3.9/site-packages (from -r requirements.txt (line 1)) (2.0.1) Requirement already satisfied: torchvision==0.15.2 in /data/eas/ENV/lib/python3.9/site-packages (from -r requirements.txt (line 2)) (0.15.2) Requirement already satisfied: diffusers==0.28.0 in /data/eas/ENV/lib/python3.9/site-packages (from -r requirements.txt (line 3)) (0.28.0) Requirement already satisfied: imageio==2.34.0 in /data/eas/ENV/lib/python3.9/site-packages (from -r requirements.txt (line 4)) (2.34.0) Requirement already satisfied: imageio-ffmpeg==0.4.9 in /data/eas/ENV/lib/python3.9/site-packages (from -r requirements.txt (line 5)) (0.4.9) Requirement already satisfied: transformers==4.40.0 in /data/eas/ENV/lib/python3.9/site-packages (from -r requirements.txt (line 6)) (4.40.0) Requirement already satisfied: numpy==1.26.4 in /data/eas/ENV/lib/python3.9/site-packages (from -r requirements.txt (line 7)) (1.26.4) Requirement already satisfied: einops==0.7.0 in /data/eas/ENV/lib/python3.9/site-packages (from -r requirements.txt (line 8)) (0.7.0) Requirement already satisfied: pillow==10.3.0 in /data/eas/ENV/lib/python3.9/site-packages (from -r requirements.txt (line 9)) (10.3.0) Requirement already satisfied: timm==1.0.7 in /data/eas/ENV/lib/python3.9/site-packages (from -r requirements.txt (line 10)) (1.0.7) Requirement already satisfied: tokenizers==0.19.1 in /data/eas/ENV/lib/python3.9/site-packages (from -r requirements.txt (line 11)) (0.19.1) Requirement already satisfied: triton==2.0.0 in /data/eas/ENV/lib/python3.9/site-packages (from -r requirements.txt (line 12)) (2.0.0) Requirement already satisfied: typing_extensions==4.11.0 in /data/eas/ENV/lib/python3.9/site-packages (from -r requirements.txt (line 13)) (4.11.0) Requirement already satisfied: torchsde==0.2.6 in /data/eas/ENV/lib/python3.9/site-packages (from -r requirements.txt (line 14)) (0.2.6) Requirement already satisfied: tqdm==4.66.2 in /data/eas/ENV/lib/python3.9/site-packages (from -r requirements.txt (line 15)) (4.66.2) Requirement already satisfied: pytorch-lightning==2.2.2 in /data/eas/ENV/lib/python3.9/site-packages (from -r requirements.txt (line 16)) (2.2.2) Requirement already satisfied: jsonargparse==4.28.0 in /data/eas/ENV/lib/python3.9/site-packages (from -r requirements.txt (line 17)) (4.28.0) Requirement already satisfied: matplotlib==3.8.4 in /data/eas/ENV/lib/python3.9/site-packages (from -r requirements.txt (line 18)) (3.8.4) Requirement already satisfied: omegaconf==2.3.0 in /data/eas/ENV/lib/python3.9/site-packages (from -r requirements.txt (line 19)) (2.3.0) Requirement already satisfied: typeshed_client==2.5.1 in /data/eas/ENV/lib/python3.9/site-packages (from -r requirements.txt (line 20)) (2.5.1) Requirement already satisfied: docstring_parser==0.16 in /data/eas/ENV/lib/python3.9/site-packages (from -r requirements.txt (line 21)) (0.16) Requirement already satisfied: kornia==0.7.2 in /data/eas/ENV/lib/python3.9/site-packages (from -r requirements.txt (line 22)) (0.7.2) Requirement already satisfied: open-clip-torch==2.24.0 in /data/eas/ENV/lib/python3.9/site-packages (from -r requirements.txt (line 23)) (2.24.0) Requirement already satisfied: xformers==0.0.20 in /data/eas/ENV/lib/python3.9/site-packages (from -r requirements.txt (line 24)) (0.0.20) Requirement already satisfied: opencv-python==4.10.0.84 in /data/eas/ENV/lib/python3.9/site-packages (from -r requirements.txt (line 25)) (4.10.0.84) Requirement already satisfied: accelerate==0.29.3 in /data/eas/ENV/lib/python3.9/site-packages (from -r requirements.txt (line 26)) (0.29.3) Requirement already satisfied: gdown==5.2.0 in /data/eas/ENV/lib/python3.9/site-packages (from -r requirements.txt (line 27)) (5.2.0) Requirement already satisfied: filelock in /data/eas/ENV/lib/python3.9/site-packages (from torch==2.0.1->-r requirements.txt (line 1)) (3.15.4) Requirement already satisfied: sympy in /data/eas/ENV/lib/python3.9/site-packages (from torch==2.0.1->-r requirements.txt (line 1)) (1.13.2) Requirement already satisfied: networkx in /data/eas/ENV/lib/python3.9/site-packages (from torch==2.0.1->-r requirements.txt (line 1)) (3.2.1) Requirement already satisfied: jinja2 in /data/eas/ENV/lib/python3.9/site-packages (from torch==2.0.1->-r requirements.txt (line 1)) (3.1.4) Requirement already satisfied: nvidia-cuda-nvrtc-cu11==11.7.99 in /data/eas/ENV/lib/python3.9/site-packages (from torch==2.0.1->-r requirements.txt (line 1)) (11.7.99) Requirement already satisfied: nvidia-cuda-runtime-cu11==11.7.99 in /data/eas/ENV/lib/python3.9/site-packages (from torch==2.0.1->-r requirements.txt (line 1)) (11.7.99) Requirement already satisfied: nvidia-cuda-cupti-cu11==11.7.101 in /data/eas/ENV/lib/python3.9/site-packages (from torch==2.0.1->-r requirements.txt (line 1)) (11.7.101) Requirement already satisfied: nvidia-cudnn-cu11==8.5.0.96 in /data/eas/ENV/lib/python3.9/site-packages (from torch==2.0.1->-r requirements.txt (line 1)) (8.5.0.96) Requirement already satisfied: nvidia-cublas-cu11==11.10.3.66 in /data/eas/ENV/lib/python3.9/site-packages (from torch==2.0.1->-r requirements.txt (line 1)) (11.10.3.66) Requirement already satisfied: nvidia-cufft-cu11==10.9.0.58 in /data/eas/ENV/lib/python3.9/site-packages (from torch==2.0.1->-r requirements.txt (line 1)) (10.9.0.58) Requirement already satisfied: nvidia-curand-cu11==10.2.10.91 in /data/eas/ENV/lib/python3.9/site-packages (from torch==2.0.1->-r requirements.txt (line 1)) (10.2.10.91) Requirement already satisfied: nvidia-cusolver-cu11==11.4.0.1 in /data/eas/ENV/lib/python3.9/site-packages (from torch==2.0.1->-r requirements.txt (line 1)) (11.4.0.1) Requirement already satisfied: nvidia-cusparse-cu11==11.7.4.91 in /data/eas/ENV/lib/python3.9/site-packages (from torch==2.0.1->-r requirements.txt (line 1)) (11.7.4.91) Requirement already satisfied: nvidia-nccl-cu11==2.14.3 in /data/eas/ENV/lib/python3.9/site-packages (from torch==2.0.1->-r requirements.txt (line 1)) (2.14.3) Requirement already satisfied: nvidia-nvtx-cu11==11.7.91 in /data/eas/ENV/lib/python3.9/site-packages (from torch==2.0.1->-r requirements.txt (line 1)) (11.7.91) Requirement already satisfied: requests in /data/eas/ENV/lib/python3.9/site-packages (from torchvision==0.15.2->-r requirements.txt (line 2)) (2.23.0) Requirement already satisfied: importlib-metadata in /data/eas/ENV/lib/python3.9/site-packages (from diffusers==0.28.0->-r requirements.txt (line 3)) (8.4.0) Requirement already satisfied: huggingface-hub>=0.20.2 in /data/eas/ENV/lib/python3.9/site-packages (from diffusers==0.28.0->-r requirements.txt (line 3)) (0.24.6) Requirement already satisfied: regex!=2019.12.17 in /data/eas/ENV/lib/python3.9/site-packages (from diffusers==0.28.0->-r requirements.txt (line 3)) (2024.7.24) Requirement already satisfied: safetensors>=0.3.1 in /data/eas/ENV/lib/python3.9/site-packages (from diffusers==0.28.0->-r requirements.txt (line 3)) (0.4.4) Requirement already satisfied: setuptools in /data/eas/ENV/lib/python3.9/site-packages (from imageio-ffmpeg==0.4.9->-r requirements.txt (line 5)) (72.1.0) Requirement already satisfied: packaging>=20.0 in /data/eas/ENV/lib/python3.9/site-packages (from transformers==4.40.0->-r requirements.txt (line 6)) (24.1) Requirement already satisfied: pyyaml>=5.1 in /data/eas/ENV/lib/python3.9/site-packages (from transformers==4.40.0->-r requirements.txt (line 6)) (6.0.2) Requirement already satisfied: cmake in /data/eas/ENV/lib/python3.9/site-packages (from triton==2.0.0->-r requirements.txt (line 12)) (3.30.3) Requirement already satisfied: lit in /data/eas/ENV/lib/python3.9/site-packages (from triton==2.0.0->-r requirements.txt (line 12)) (18.1.8) Requirement already satisfied: scipy>=1.5 in /data/eas/ENV/lib/python3.9/site-packages (from torchsde==0.2.6->-r requirements.txt (line 14)) (1.13.1) Requirement already satisfied: trampoline>=0.1.2 in /data/eas/ENV/lib/python3.9/site-packages (from torchsde==0.2.6->-r requirements.txt (line 14)) (0.1.2) Requirement already satisfied: fsspec>=2022.5.0 in /data/eas/ENV/lib/python3.9/site-packages (from fsspec[http]>=2022.5.0->pytorch-lightning==2.2.2->-r requirements.txt (line 16)) (2024.6.1) Requirement already satisfied: torchmetrics>=0.7.0 in /data/eas/ENV/lib/python3.9/site-packages (from pytorch-lightning==2.2.2->-r requirements.txt (line 16)) (1.4.2) Requirement already satisfied: lightning-utilities>=0.8.0 in /data/eas/ENV/lib/python3.9/site-packages (from pytorch-lightning==2.2.2->-r requirements.txt (line 16)) (0.11.7) Requirement already satisfied: contourpy>=1.0.1 in /data/eas/ENV/lib/python3.9/site-packages (from matplotlib==3.8.4->-r requirements.txt (line 18)) (1.3.0) Requirement already satisfied: cycler>=0.10 in /data/eas/ENV/lib/python3.9/site-packages (from matplotlib==3.8.4->-r requirements.txt (line 18)) (0.12.1) Requirement already satisfied: fonttools>=4.22.0 in /data/eas/ENV/lib/python3.9/site-packages (from matplotlib==3.8.4->-r requirements.txt (line 18)) (4.53.1) Requirement already satisfied: kiwisolver>=1.3.1 in /data/eas/ENV/lib/python3.9/site-packages (from matplotlib==3.8.4->-r requirements.txt (line 18)) (1.4.5) Requirement already satisfied: pyparsing>=2.3.1 in /data/eas/ENV/lib/python3.9/site-packages (from matplotlib==3.8.4->-r requirements.txt (line 18)) (3.1.4) Requirement already satisfied: python-dateutil>=2.7 in /data/eas/ENV/lib/python3.9/site-packages (from matplotlib==3.8.4->-r requirements.txt (line 18)) (2.9.0.post0) Requirement already satisfied: importlib-resources>=3.2.0 in /data/eas/ENV/lib/python3.9/site-packages (from matplotlib==3.8.4->-r requirements.txt (line 18)) (6.4.4) Requirement already satisfied: antlr4-python3-runtime==4.9.* in /data/eas/ENV/lib/python3.9/site-packages (from omegaconf==2.3.0->-r requirements.txt (line 19)) (4.9.3) Requirement already satisfied: kornia-rs>=0.1.0 in /data/eas/ENV/lib/python3.9/site-packages (from kornia==0.7.2->-r requirements.txt (line 22)) (0.1.5) Requirement already satisfied: ftfy in /data/eas/ENV/lib/python3.9/site-packages (from open-clip-torch==2.24.0->-r requirements.txt (line 23)) (6.2.3) Requirement already satisfied: sentencepiece in /data/eas/ENV/lib/python3.9/site-packages (from open-clip-torch==2.24.0->-r requirements.txt (line 23)) (0.2.0) Requirement already satisfied: protobuf in /data/eas/ENV/lib/python3.9/site-packages (from open-clip-torch==2.24.0->-r requirements.txt (line 23)) (5.28.0) Requirement already satisfied: pyre-extensions==0.0.29 in /data/eas/ENV/lib/python3.9/site-packages (from xformers==0.0.20->-r requirements.txt (line 24)) (0.0.29) Requirement already satisfied: psutil in /data/eas/ENV/lib/python3.9/site-packages (from accelerate==0.29.3->-r requirements.txt (line 26)) (6.0.0) Requirement already satisfied: beautifulsoup4 in /data/eas/ENV/lib/python3.9/site-packages (from gdown==5.2.0->-r requirements.txt (line 27)) (4.12.3) Requirement already satisfied: wheel in /data/eas/ENV/lib/python3.9/site-packages (from nvidia-cublas-cu11==11.10.3.66->torch==2.0.1->-r requirements.txt (line 1)) (0.43.0) Requirement already satisfied: typing-inspect in /data/eas/ENV/lib/python3.9/site-packages (from pyre-extensions==0.0.29->xformers==0.0.20->-r requirements.txt (line 24)) (0.9.0) Requirement already satisfied: aiohttp!=4.0.0a0,!=4.0.0a1 in /data/eas/ENV/lib/python3.9/site-packages (from fsspec[http]>=2022.5.0->pytorch-lightning==2.2.2->-r requirements.txt (line 16)) (3.10.6) Requirement already satisfied: zipp>=3.1.0 in /data/eas/ENV/lib/python3.9/site-packages (from importlib-resources>=3.2.0->matplotlib==3.8.4->-r requirements.txt (line 18)) (3.20.1) Requirement already satisfied: six>=1.5 in /data/eas/ENV/lib/python3.9/site-packages (from python-dateutil>=2.7->matplotlib==3.8.4->-r requirements.txt (line 18)) (1.16.0) Requirement already satisfied: soupsieve>1.2 in /data/eas/ENV/lib/python3.9/site-packages (from beautifulsoup4->gdown==5.2.0->-r requirements.txt (line 27)) (2.6) Requirement already satisfied: wcwidth<0.3.0,>=0.2.12 in /data/eas/ENV/lib/python3.9/site-packages (from ftfy->open-clip-torch==2.24.0->-r requirements.txt (line 23)) (0.2.13) Requirement already satisfied: MarkupSafe>=2.0 in /data/eas/ENV/lib/python3.9/site-packages (from jinja2->torch==2.0.1->-r requirements.txt (line 1)) (2.1.5) Requirement already satisfied: chardet<4,>=3.0.2 in /data/eas/ENV/lib/python3.9/site-packages (from requests->torchvision==0.15.2->-r requirements.txt (line 2)) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /data/eas/ENV/lib/python3.9/site-packages (from requests->torchvision==0.15.2->-r requirements.txt (line 2)) (2.10) Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /data/eas/ENV/lib/python3.9/site-packages (from requests->torchvision==0.15.2->-r requirements.txt (line 2)) (1.23) Requirement already satisfied: certifi>=2017.4.17 in /data/eas/ENV/lib/python3.9/site-packages (from requests->torchvision==0.15.2->-r requirements.txt (line 2)) (2024.7.4) Requirement already satisfied: PySocks!=1.5.7,>=1.5.6 in /data/eas/ENV/lib/python3.9/site-packages (from requests[socks]->gdown==5.2.0->-r requirements.txt (line 27)) (1.7.1) Requirement already satisfied: mpmath<1.4,>=1.1.0 in /data/eas/ENV/lib/python3.9/site-packages (from sympy->torch==2.0.1->-r requirements.txt (line 1)) (1.3.0) Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /data/eas/ENV/lib/python3.9/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>=2022.5.0->pytorch-lightning==2.2.2->-r requirements.txt (line 16)) (2.4.2) Requirement already satisfied: aiosignal>=1.1.2 in /data/eas/ENV/lib/python3.9/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>=2022.5.0->pytorch-lightning==2.2.2->-r requirements.txt (line 16)) (1.3.1) Requirement already satisfied: attrs>=17.3.0 in /data/eas/ENV/lib/python3.9/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>=2022.5.0->pytorch-lightning==2.2.2->-r requirements.txt (line 16)) (24.2.0) Requirement already satisfied: frozenlist>=1.1.1 in /data/eas/ENV/lib/python3.9/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>=2022.5.0->pytorch-lightning==2.2.2->-r requirements.txt (line 16)) (1.4.1) Requirement already satisfied: multidict<7.0,>=4.5 in /data/eas/ENV/lib/python3.9/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>=2022.5.0->pytorch-lightning==2.2.2->-r requirements.txt (line 16)) (6.1.0) Requirement already satisfied: yarl<2.0,>=1.12.0 in /data/eas/ENV/lib/python3.9/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>=2022.5.0->pytorch-lightning==2.2.2->-r requirements.txt (line 16)) (1.13.0) Requirement already satisfied: async-timeout<5.0,>=4.0 in /data/eas/ENV/lib/python3.9/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>=2022.5.0->pytorch-lightning==2.2.2->-r requirements.txt (line 16)) (4.0.3) Requirement already satisfied: mypy-extensions>=0.3.0 in /data/eas/ENV/lib/python3.9/site-packages (from typing-inspect->pyre-extensions==0.0.29->xformers==0.0.20->-r requirements.txt (line 24)) (1.0.0) WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning. root@a2f93364212e:/mnt/host/data/StreamingT2V# pip freeze >> requirements.txt

[jiyuwangbupt]

Hi @jiyuwangbupt ,

please check this issue: #59

Due to my location in mainland China, I am unable to automatically download the Hugging Face model weights. Therefore, I downloaded them manually and stored them in a local path. I changed the model path to an absolute path and modified the pretrained_model path in diffusers/pipelines/pipeline_utils.py to an absolute path as well. I'm not sure if this will have any impact.

[rob-hen]

Hi @jiyuwangbupt,

unfortunately, I cannot reproduce your issue. However, it should be related to jsonargparse. Can you try different, or newer versions of jsonargparse, and check if that helps? To understand when its correct, you should not get your message Warning: Seems object conditioner was not build correct.