search

fudan-generative-vision / champ

Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance
https://fudan-generative-vision.github.io/champ/
MIT License
4.75k stars 598 forks source link

_pickle.UnpicklingError: Weights only load failed. #144

Closed j-yi-11 closed 3 months ago

j-yi-11 commented 4 months ago

复现时候遇到的一些问题

作者您好，我对您所做的champ的工作非常感兴趣，但是我在RTX4090上进行复现的时候出现了一些问题。

运行命令

python inference.py --config configs/inference/inference.yaml

inference.yaml内容如下：

exp_name: Animation
width: 512
height: 512
data:
  ref_image_path: 'example_data/ref_images/ref-02.png'  # reference image path
  guidance_data_folder: 'example_data/motions/motion-02'  # corresponding motion sequence folder
  frame_range: [0, 100]  # [Optional] specify a frame range: [min_frame_idx, max_frame_idx] to select a clip from a motion sequence  
seed: 42

base_model_path: 'pretrained_models/stable-diffusion-v1-5'
vae_model_path: 'pretrained_models/sd-vae-ft-mse'
image_encoder_path: 'pretrained_models/image_encoder'

ckpt_dir: 'pretrained_models/champ'
motion_module_path: 'pretrained_models/champ/motion_module.pth'

num_inference_steps: 20
guidance_scale: 3.5
enable_zero_snr: true
weight_dtype: "fp16"

guidance_types:
  - 'depth'
  - 'normal'
  - 'semantic_map'
  - 'dwpose'

noise_scheduler_kwargs:
  num_train_timesteps: 1000
  beta_start:          0.00085
  beta_end:            0.012
  beta_schedule:       "linear"
  steps_offset:        1
  clip_sample:         false

unet_additional_kwargs:
  use_inflated_groupnorm: true
  unet_use_cross_frame_attention: false 
  unet_use_temporal_attention: false
  use_motion_module: true
  motion_module_resolutions:
  - 1
  - 2
  - 4
  - 8
  motion_module_mid_block: true 
  motion_module_decoder_only: false
  motion_module_type: Vanilla
  motion_module_kwargs:
    num_attention_heads: 8
    num_transformer_block: 1
    attention_block_types:
    - Temporal_Self
    - Temporal_Self
    temporal_position_encoding: true
    temporal_position_encoding_max_len: 32
    temporal_attention_dim_div: 1

guidance_encoder_kwargs:
  guidance_embedding_channels: 320
  guidance_input_channels: 3
  block_out_channels: [16, 32, 96, 256]

enable_xformers_memory_efficient_attention: true

环境中包的信息

按照requirements.txt进行安装得到：

pip list                                                      
Package                        Version
------------------------------ --------------------
absl-py                        2.1.0
accelerate                     0.21.0
antlr4-python3-runtime         4.9.3
certifi                        2024.7.4
charset-normalizer             3.3.2
coloredlogs                    15.0.1
contourpy                      1.2.1
controlnet_aux                 0.0.7
cycler                         0.12.1
diffusers                      0.24.0
einops                         0.4.1
filelock                       3.15.4
flatbuffers                    24.3.25
fonttools                      4.53.1
freetype-py                    2.4.0
fsspec                         2024.6.1
ftfy                           6.2.0
grpcio                         1.64.1
huggingface-hub                0.23.4
humanfriendly                  10.0
idna                           3.7
imageio                        2.33.0
imageio-ffmpeg                 0.5.1
importlib_metadata             8.0.0
Jinja2                         3.1.4
kiwisolver                     1.4.5
lazy_loader                    0.4
lightning-utilities            0.11.3.post0
Markdown                       3.6
MarkupSafe                     2.1.5
matplotlib                     3.8.3
mpmath                         1.3.0
networkx                       3.3
numpy                          1.23.5
nvidia-cublas-cu12             12.1.3.1
nvidia-cuda-cupti-cu12         12.1.105
nvidia-cuda-nvrtc-cu12         12.1.105
nvidia-cuda-runtime-cu12       12.1.105
nvidia-cudnn-cu12              8.9.2.26
nvidia-cufft-cu12              11.0.2.54
nvidia-curand-cu12             10.3.2.106
nvidia-cusolver-cu12           11.4.5.107
nvidia-cusparse-cu12           12.1.0.106
nvidia-nccl-cu12               2.19.3
nvidia-nvjitlink-cu12          12.5.82
nvidia-nvtx-cu12               12.1.105
omegaconf                      2.2.3
onnxruntime-gpu                1.13.1
open-clip-torch                2.20.0
opencv-contrib-python-headless 4.8.1.78
opencv-python                  4.10.0.84
opencv-python-headless         4.8.1.78
packaging                      24.1
pandas                         2.2.2
Pillow                         9.5.0
pip                            24.0
protobuf                       3.20.3
psutil                         6.0.0
py-cpuinfo                     9.0.0
pyglet                         2.0.15
PyOpenGL                       3.1.0
pyparsing                      3.1.2
pyrender                       0.1.45
python-dateutil                2.9.0.post0
pytz                           2024.1
PyYAML                         6.0.1
regex                          2024.5.15
requests                       2.32.3
safetensors                    0.4.2
scikit-image                   0.24.0
scipy                          1.14.0
seaborn                        0.13.2
sentencepiece                  0.2.0
setuptools                     70.1.1
six                            1.16.0
sympy                          1.12.1
tensorboard                    2.17.0
tensorboard-data-server        0.7.2
thop                           0.1.1.post2209072238
tifffile                       2024.7.2
timm                           1.0.7
tokenizers                     0.13.3
torch                          2.2.2
torchmetrics                   1.2.1
torchvision                    0.17.2
tqdm                           4.66.1
transformers                   4.30.2
trimesh                        4.4.1
triton                         2.2.0
typing_extensions              4.12.2
tzdata                         2024.1
ultralytics                    8.1.34
urllib3                        2.2.2
wcwidth                        0.2.13
Werkzeug                       3.0.3
wheel                          0.43.0
xformers                       0.0.25.post1
zipp                           3.19.2

终端输出

从huggingface上的主页fudan-generative-ai/champ下载pretrained_models后，运行python inference.py --config configs/inference/inference.yaml

/home/xxj/anaconda3/envs/champ/lib/python3.10/site-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  torch.utils._pytree._register_pytree_node(
07/09/2024 12:44:14 - INFO - root - Running inference ...
07/09/2024 12:44:29 - INFO - models.unet_3d - loaded temporal unet's pretrained weights from pretrained_models/stable-diffusion-v1-5/unet ...
Traceback (most recent call last):
  File "/data/champ/inference.py", line 319, in <module>
    main(cfg)
  File "/data/champ/inference.py", line 192, in main
    denoising_unet = UNet3DConditionModel.from_pretrained_2d(
  File "/data/champ/models/unet_3d.py", line 626, in from_pretrained_2d
    state_dict = torch.load(
  File "/home/xxj/anaconda3/envs/champ/lib/python3.10/site-packages/torch/serialization.py", line 1025, in load
    raise pickle.UnpicklingError(UNSAFE_MESSAGE + str(e)) from None
_pickle.UnpicklingError: Weights only load failed. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution.Do it only if you get the file from a trusted source. WeightsUnpickler error: PytorchStreamReader failed reading file data/2: invalid header or archive is corrupted
AricGamma commented 4 months ago

Please check directory structure under pretrained_models/stable-diffusion-v1-5/, and make sure that every model has the same hash with huggingface ones.

AricGamma commented 3 months ago

Closing this issue. If any further questions, please reopen it.