THUDM / CogVideo

text and image to video generation: CogVideoX (2024) and CogVideo (ICLR 2023)
Apache License 2.0
8.4k stars 803 forks source link

[rank0]: OSError: t5-v1_1-xxl is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models' #209

Closed Tianyu9748 closed 1 month ago

Tianyu9748 commented 2 months ago

Thank you for the open source and the guidance. I encountered some problems when generating the video.

CogVideoX-2b ├── LICENSE ├── model_index.json ├── README.md ├── README_zh.md ├── scheduler │ └── scheduler_config.json ├── t5-v1_1-xxl │ ├── added_tokens.json │ ├── CogVideoX-2b-sat │ │ ├── transformer │ │ │ ├── 1000 │ │ │ │ └── mp_rank_00_model_states.pt │ │ │ └── latest │ │ ├── transformer.zip │ │ ├── vae │ │ │ └── 3d-vae.pt │ │ └── vae.zip │ ├── config.json │ ├── model-00001-of-00002.safetensors │ ├── model-00002-of-00002.safetensors │ ├── model.safetensors.index.json │ ├── special_tokens_map.json │ ├── spiece.model │ └── tokenizer_config.json ├── text_encoder ├── tokenizer ├── transformer │ ├── config.json │ └── diffusion_pytorch_model.safetensors └── vae ├── config.json └── diffusion_pytorch_model.safetensors

I updated the yaml files as this instruction.

How to solve this?

Best Tianyu

zRzRzRzRzRzRzR commented 2 months ago

In our SAT, it is mentioned that the corresponding T5 module should be downloaded from HuggingFace for CogVideoX2B / 5B. The T5 module is loaded separately. You can refer to our SAT's README for configuration instructions.

Sang-Yeop-Yeo commented 1 month ago

I face the same issue as well.

Have you resolved it?

Is the correct path for t5-v1_1-xxl CogVideo/t5-v1_1-xxl? Is that right?

zRzRzRzRzRzRzR commented 1 month ago

The T5 model needs to be extracted separately from the CogVideoX diffusers version and integrated into a new file, or you can use another T5 in the safetensor format. This T5 only contains the encoder part.

Tianyu9748 commented 1 month ago

Thank you for the help. The issuse is solved.