search

PKU-YuanGroup / Open-Sora-Plan

This project aim to reproduce Sora (Open AI T2V model), we wish the open source community contribute to this project.
MIT License
10.95k stars 976 forks source link

Failed loading VAE checkpoint #273

Open ROGERDJQ opened 2 months ago

ROGERDJQ commented 2 months ago

Thank you for your wonderful contributions! I have some issues with the loading of VAE checkpoint. when loading the checkpoint (https://huggingface.co/LanguageBind/Open-Sora-Plan-v1.0.0/tree/main/vae), we found that code vae = CausalVAEModel.from_pretrained('./ckpt/Open-Sora-Plan-v1.0.0/vae/') raises error UnpicklingError: invalid load key, '\xb0'.

We also try another solution as vae = CausalVAEModel.from_config('./ckpt/Open-Sora-Plan-v1.0.0/vae/config.json') model = vae.init_from_ckpt('./ckpt/Open-Sora-Plan-v1.0.0/vae/diffusion_pytorch_model.safetensors') failed again with same error.

Additionally, directly applying pickle returns the same error. So what could be the reason for that? and how to load the checkpoint?

LinB203 commented 1 month ago

CausalVAEModel.from_pretrained('./ckpt/Open-Sora-Plan-v1.0.0/vae/'). It should be a config.json and weight under the folder.