search

IanYeung / MGLD-VSR

Code for ECCV 2024 Paper "Motion-Guided Latent Diffusion for Temporally Consistent Real-world Video Super-resolution"
Other
110 stars 3 forks source link

ModuleNotFoundError: No module named 'ldm' #2

Open zelenooki87 opened 11 months ago

zelenooki87 commented 11 months ago

Thank you for open sourcing code. I have a problem during testing:

python scripts/vsr_val_ddpm_text_T_vqganfin_oldcanvas_tile.py  --config configs/mgldvsr/mgldvsr_512_realbasicvsr_deg.yaml  --ckpt "C:\Users\miki\MGLD-VSR\mgldvsr_unet.ckpt"  --vqgan_ckpt "C:\Users\miki\MGLD-VSR\video_vae_cfw.ckpt"  --seqs-path "C:\Users\miki\MGLD-VSR\input"   --outdir "C:\Users\miki\MGLD-VSR\out"  --ddpm_steps 50  --dec_w 1.0  --colorfix_type adain  --select_idx 0  --n_gpus 1
Traceback (most recent call last):
  File "C:\Users\miki\MGLD-VSR\scripts\vsr_val_ddpm_text_T_vqganfin_oldcanvas_tile.py", line 19, in <module>
    from ldm.util import instantiate_from_config
ModuleNotFoundError: **No module named 'ldm'**

Same error on the Ubuntu and Windows. installing ldm through pypi doesnt help at all cause latest version of ldm is for python v2...

IanYeung commented 11 months ago

Thank you for open sourcing code. I have a problem during testing:

python scripts/vsr_val_ddpm_text_T_vqganfin_oldcanvas_tile.py  --config configs/mgldvsr/mgldvsr_512_realbasicvsr_deg.yaml  --ckpt "C:\Users\miki\MGLD-VSR\mgldvsr_unet.ckpt"  --vqgan_ckpt "C:\Users\miki\MGLD-VSR\video_vae_cfw.ckpt"  --seqs-path "C:\Users\miki\MGLD-VSR\input"   --outdir "C:\Users\miki\MGLD-VSR\out"  --ddpm_steps 50  --dec_w 1.0  --colorfix_type adain  --select_idx 0  --n_gpus 1
Traceback (most recent call last):
  File "C:\Users\miki\MGLD-VSR\scripts\vsr_val_ddpm_text_T_vqganfin_oldcanvas_tile.py", line 19, in <module>
    from ldm.util import instantiate_from_config
ModuleNotFoundError: **No module named 'ldm'**

Same error on the Ubuntu and Windows. installing ldm through pypi doesnt help at all cause latest version of ldm is for python v2...

Maybe you can try the following:

export PYTHONPATH=".:$PYTHONPATH"
zelenooki87 commented 11 months ago

Okay. I had to install mmcv. During inference this error now I got: `python scripts/vsr_val_ddpm_text_T_vqganfin_oldcanvas_tile.py --config configs/mgldvsr/mgldvsr_512_realbasicvsr_deg.yaml --ckpt "C:\Users\miki\MGLD-VSR\mgldvsr_unet.ckpt" --vqgan_ckpt "C:\Users\miki\MGLD-VSR\video_vae_cfw.ckpt" --seqs-path "C:\Users\miki\MGLD-VSR\input" --outdir "C:\Users\miki\MGLD-VSR\out" --ddpm_steps 50 --dec_w 1.0 --colorfix_type adain --select_idx 0 --n_gpus 1 Global seed set to 42

color correction>>>>>>>>>>> Use adain color correction

Loading model from C:\Users\miki\MGLD-VSR\mgldvsr_unet.ckpt Global Step: 42000 LatentDiffusionVSRTextWT: Running in eps-prediction mode Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 5 heads. Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 1024 and using 5 heads. Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 5 heads. Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 1024 and using 5 heads. Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 10 heads. Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 1024 and using 10 heads. Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 10 heads. Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 1024 and using 10 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 20 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 1024 and using 20 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 20 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 1024 and using 20 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 20 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 1024 and using 20 heads. Setting up MemoryEfficientSelfAttention. Query dim is 1280, using 20 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 20 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 1024 and using 20 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 20 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 1024 and using 20 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 20 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 1024 and using 20 heads. Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 10 heads. Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 1024 and using 10 heads. Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 10 heads. Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 1024 and using 10 heads. Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 10 heads. Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 1024 and using 10 heads. Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 5 heads. Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 1024 and using 5 heads. Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 5 heads. Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 1024 and using 5 heads. Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 5 heads. Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 1024 and using 5 heads. DiffusionWrapper has 935.32 M params. making attention of type 'vanilla' with 512 in_channels Working with z of shape (1, 4, 64, 64) = 16384 dimensions. making attention of type 'vanilla' with 512 in_channels Traceback (most recent call last): File "C:\Users\miki\MGLD-VSR\scripts\vsr_val_ddpm_text_T_vqganfin_oldcanvas_tile.py", line 554, in main() File "C:\Users\miki\MGLD-VSR\scripts\vsr_val_ddpm_text_T_vqganfin_oldcanvas_tile.py", line 297, in main model = load_model_from_config(config, f"{opt.ckpt}") File "C:\Users\miki\MGLD-VSR\scripts\vsr_val_ddpm_text_T_vqganfin_oldcanvas_tile.py", line 97, in load_model_from_config model = instantiate_from_config(config.model) File "C:\Users\miki\MGLD-VSR\ldm\util.py", line 85, in instantiate_from_config return get_obj_from_str(config["target"])(config.get("params", dict())) File "C:\Users\miki\MGLD-VSR\ldm\models\diffusion\ddpm.py", line 3223, in init self.instantiate_first_stage(first_stage_config) File "C:\Users\miki\MGLD-VSR\ldm\models\diffusion\ddpm.py", line 3329, in instantiate_first_stage model = instantiate_from_config(config) File "C:\Users\miki\MGLD-VSR\ldm\util.py", line 85, in instantiate_from_config return get_obj_from_str(config["target"])(config.get("params", dict())) File "C:\Users\miki\MGLD-VSR\ldm\models\autoencoder.py", line 325, in init self.init_from_ckpt(ckpt_path, ignore_keys=ignore_keys) File "C:\Users\miki\MGLD-VSR\ldm\models\autoencoder.py", line 328, in init_from_ckpt sd = torch.load(path, map_location="cpu") File "C:\Users\miki\anaconda3\envs\vsr\lib\site-packages\torch\serialization.py", line 771, in load with _open_file_like(f, 'rb') as opened_file: File "C:\Users\miki\anaconda3\envs\vsr\lib\site-packages\torch\serialization.py", line 270, in _open_file_like return _open_file(name_or_buffer, mode) File "C:\Users\miki\anaconda3\envs\vsr\lib\site-packages\torch\serialization.py", line 251, in init super(_open_file, self).init(open(name, mode)) FileNotFoundError: [Errno 2] No such file or directory: '/home/notebook/code/personal/xxxxxxxx/MGLD-VSR/checkpoints/v2-1_512-ema-pruned.ckpt'` please help

zelenooki87 commented 11 months ago

okay, downloaded v2-1_512-ema-pruned.ckpt and open_clip_pytorch_model.bin models and change path in mgldvsr_512_realbasicvsr_deg.yaml to them path.... without success... i will probably give up if you do not help thanks!

meisa233 commented 11 months ago

This is a documentation about some solutions to resolve errors when I run the script. But I don't have enough memory, I can't run it successfully. https://gist.github.com/meisa233/1549bb95c5c130e3a93fcab17c83e931

fzuqxt commented 10 months ago

Is there a download link for open_clip_pytorch_model.bin？

meisa233 commented 10 months ago

Is there a download link for open_clip_pytorch_model.bin？

https://huggingface.co/laion/CLIP-ViT-H-14-laion2B-s32B-b79K/tree/main

fzuqxt commented 10 months ago

@meisa233 thanks ！Did you run it successfully

meisa233 commented 10 months ago

@meisa233 thanks ！Did you run it successfully

I don't have enough gpu memory, so i can't run it.

fzuqxt commented 10 months ago

At least how much memory does this need？

meisa233 commented 10 months ago

At least how much memory does this need？

My GPU memory is 24G, which is not enough. The model described in this paper is trained on NVIDIA A100.

gyfastas commented 9 months ago

I think it is better to have a setup.py. For example, following https://github.com/IceClear/StableSR/blob/main/setup.py

leejielong commented 9 months ago

@meisa233 I managed to run it within 24GB GPU memory. The problem arises when we try to decode using the temporal aware sequence decoder (vqgan decoder). The solution I found was to reduce --vqgantile_size to 512 instead of 960.

hhuyujie commented 9 months ago

hello, do you have such problem: image Can someone help me with this?

alasokolova commented 6 months ago

This is a documentation about some solutions to resolve errors when I run the script. But I don't have enough memory, I can't run it successfully. https://gist.github.com/meisa233/1549bb95c5c130e3a93fcab17c83e931

@meisa233 Thanks for this documentation! I have downloaded all the models but when a load them it still says that there are some missing parameters. Have you faced it?

meisa233 commented 6 months ago

This is a documentation about some solutions to resolve errors when I run the script. But I don't have enough memory, I can't run it successfully. https://gist.github.com/meisa233/1549bb95c5c130e3a93fcab17c83e931

@meisa233 Thanks for this documentation! I have downloaded all the models but when a load them it still says that there are some missing parameters. Have you faced it?

I haven't encountered it. Which missing parameters it showed?

alasokolova commented 6 months ago

Which missing parameters it showed?

Encoder Restored from v2-1_512-ema-pruned.ckpt with 230 missing and 994 unexpected keys

Missing Keys: ['decoder.temporal_mixing.temporal_alpha', 'decoder.temporal_mixing.temporal_conv.weight', 'decoder.temporal_mixing.temporal_conv.bias', 'decoder.up.0.temporal_mixing.0.temporal_alpha', 'decoder.up.0.temporal_mixing.0.temporal_conv.weight', 'decoder.up.0.temporal_mixing.0.temporal_conv.bias', 'decoder.up.0.temporal_mixing.1.temporal_alpha', 'decoder.up.0.temporal_mixing.1.temporal_conv.weight', 'decoder.up.0.temporal_mixing.1.temporal_conv.bias', 'decoder.up.0.temporal_mixing.2.temporal_alpha', 'decoder.up.0.temporal_mixing.2.temporal_conv.weight', 'decoder.up.0.temporal_mixing.2.temporal_conv.bias', 'decoder.up.1.temporal_mixing.0.temporal_alpha', 'decoder.up.1.temporal_mixing.0.temporal_conv.weight', 'decoder.up.1.temporal_mixing.0.temporal_conv.bias', 'decoder.up.1.temporal_mixing.1.temporal_alpha', 'decoder.up.1.temporal_mixing.1.temporal_conv.weight', 'decoder.up.1.temporal_mixing.1.temporal_conv.bias', 'decoder.up.1.temporal_mixing.2.temporal_alpha', 'decoder.up.1.temporal_mixing.2.temporal_conv.weight', 'decoder.up.1.temporal_mixing.2.temporal_conv.bias', 'decoder.up.2.temporal_mixing.0.temporal_alpha', 'decoder.up.2.temporal_mixing.0.temporal_conv.weight', 'decoder.up.2.temporal_mixing.0.temporal_conv.bias', 'decoder.up.2.temporal_mixing.1.temporal_alpha', 'decoder.up.2.temporal_mixing.1.temporal_conv.weight', 'decoder.up.2.temporal_mixing.1.temporal_conv.bias', 'decoder.up.2.temporal_mixing.2.temporal_alpha', 'decoder.up.2.temporal_mixing.2.temporal_conv.weight', 'decoder.up.2.temporal_mixing.2.temporal_conv.bias', 'decoder.up.3.temporal_mixing.0.temporal_alpha', 'decoder.up.3.temporal_mixing.0.temporal_conv.weight', 'decoder.up.3.temporal_mixing.0.temporal_conv.bias', 'decoder.up.3.temporal_mixing.1.temporal_alpha', 'decoder.up.3.temporal_mixing.1.temporal_conv.weight', 'decoder.up.3.temporal_mixing.1.temporal_conv.bias', 'decoder.up.3.temporal_mixing.2.temporal_alpha', 'decoder.up.3.temporal_mixing.2.temporal_conv.weight', 'decoder.up.3.temporal_mixing.2.temporal_conv.bias', 'decoder.fusion_layer_2.encode_enc_1.norm1.weight', 'decoder.fusion_layer_2.encode_enc_1.norm1.bias', 'decoder.fusion_layer_2.encode_enc_1.conv1.weight', 'decoder.fusion_layer_2.encode_enc_1.conv1.bias', 'decoder.fusion_layer_2.encode_enc_1.norm2.weight', 'decoder.fusion_layer_2.encode_enc_1.norm2.bias', 'decoder.fusion_layer_2.encode_enc_1.conv2.weight', 'decoder.fusion_layer_2.encode_enc_1.conv2.bias', 'decoder.fusion_layer_2.encode_enc_1.conv_out.weight', 'decoder.fusion_layer_2.encode_enc_1.conv_out.bias', 'decoder.fusion_layer_2.encode_enc_2.0.conv1.weight', 'decoder.fusion_layer_2.encode_enc_2.0.conv1.bias', 'decoder.fusion_layer_2.encode_enc_2.0.conv2.weight', 'decoder.fusion_layer_2.encode_enc_2.0.conv2.bias', 'decoder.fusion_layer_2.encode_enc_2.0.conv3.weight', 'decoder.fusion_layer_2.encode_enc_2.0.conv3.bias', 'decoder.fusion_layer_2.encode_enc_2.0.conv4.weight', 'decoder.fusion_layer_2.encode_enc_2.0.conv4.bias', 'decoder.fusion_layer_2.encode_enc_2.0.conv5.weight', 'decoder.fusion_layer_2.encode_enc_2.0.conv5.bias', 'decoder.fusion_layer_2.encode_enc_2.1.conv1.weight', 'decoder.fusion_layer_2.encode_enc_2.1.conv1.bias', 'decoder.fusion_layer_2.encode_enc_2.1.conv2.weight', 'decoder.fusion_layer_2.encode_enc_2.1.conv2.bias', 'decoder.fusion_layer_2.encode_enc_2.1.conv3.weight', 'decoder.fusion_layer_2.encode_enc_2.1.conv3.bias', 'decoder.fusion_layer_2.encode_enc_2.1.conv4.weight', 'decoder.fusion_layer_2.encode_enc_2.1.conv4.bias', 'decoder.fusion_layer_2.encode_enc_2.1.conv5.weight', 'decoder.fusion_layer_2.encode_enc_2.1.conv5.bias', 'decoder.fusion_layer_2.encode_enc_3.norm1.weight', 'decoder.fusion_layer_2.encode_enc_3.norm1.bias', 'decoder.fusion_layer_2.encode_enc_3.conv1.weight', 'decoder.fusion_layer_2.encode_enc_3.conv1.bias', 'decoder.fusion_layer_2.encode_enc_3.norm2.weight', 'decoder.fusion_layer_2.encode_enc_3.norm2.bias', 'decoder.fusion_layer_2.encode_enc_3.conv2.weight', 'decoder.fusion_layer_2.encode_enc_3.conv2.bias', 'decoder.fusion_layer_1.encode_enc_1.norm1.weight', 'decoder.fusion_layer_1.encode_enc_1.norm1.bias', 'decoder.fusion_layer_1.encode_enc_1.conv1.weight', 'decoder.fusion_layer_1.encode_enc_1.conv1.bias', 'decoder.fusion_layer_1.encode_enc_1.norm2.weight', 'decoder.fusion_layer_1.encode_enc_1.norm2.bias', 'decoder.fusion_layer_1.encode_enc_1.conv2.weight', 'decoder.fusion_layer_1.encode_enc_1.conv2.bias', 'decoder.fusion_layer_1.encode_enc_1.conv_out.weight', 'decoder.fusion_layer_1.encode_enc_1.conv_out.bias', 'decoder.fusion_layer_1.encode_enc_2.0.conv1.weight', 'decoder.fusion_layer_1.encode_enc_2.0.conv1.bias', 'decoder.fusion_layer_1.encode_enc_2.0.conv2.weight', 'decoder.fusion_layer_1.encode_enc_2.0.conv2.bias', 'decoder.fusion_layer_1.encode_enc_2.0.conv3.weight', 'decoder.fusion_layer_1.encode_enc_2.0.conv3.bias', 'decoder.fusion_layer_1.encode_enc_2.0.conv4.weight', 'decoder.fusion_layer_1.encode_enc_2.0.conv4.bias', 'decoder.fusion_layer_1.encode_enc_2.0.conv5.weight', 'decoder.fusion_layer_1.encode_enc_2.0.conv5.bias', 'decoder.fusion_layer_1.encode_enc_2.1.conv1.weight', 'decoder.fusion_layer_1.encode_enc_2.1.conv1.bias', 'decoder.fusion_layer_1.encode_enc_2.1.conv2.weight', 'decoder.fusion_layer_1.encode_enc_2.1.conv2.bias', 'decoder.fusion_layer_1.encode_enc_2.1.conv3.weight', 'decoder.fusion_layer_1.encode_enc_2.1.conv3.bias', 'decoder.fusion_layer_1.encode_enc_2.1.conv4.weight', 'decoder.fusion_layer_1.encode_enc_2.1.conv4.bias', 'decoder.fusion_layer_1.encode_enc_2.1.conv5.weight', 'decoder.fusion_layer_1.encode_enc_2.1.conv5.bias', 'decoder.fusion_layer_1.encode_enc_3.norm1.weight', 'decoder.fusion_layer_1.encode_enc_3.norm1.bias', 'decoder.fusion_layer_1.encode_enc_3.conv1.weight', 'decoder.fusion_layer_1.encode_enc_3.conv1.bias', 'decoder.fusion_layer_1.encode_enc_3.norm2.weight', 'decoder.fusion_layer_1.encode_enc_3.norm2.bias', 'decoder.fusion_layer_1.encode_enc_3.conv2.weight', 'decoder.fusion_layer_1.encode_enc_3.conv2.bias', 'loss.logvar', 'loss.perceptual_loss.scaling_layer.shift', 'loss.perceptual_loss.scaling_layer.scale', 'loss.perceptual_loss.net.slice1.0.weight', 'loss.perceptual_loss.net.slice1.0.bias', 'loss.perceptual_loss.net.slice1.2.weight', 'loss.perceptual_loss.net.slice1.2.bias', 'loss.perceptual_loss.net.slice2.5.weight', 'loss.perceptual_loss.net.slice2.5.bias', 'loss.perceptual_loss.net.slice2.7.weight', 'loss.perceptual_loss.net.slice2.7.bias', 'loss.perceptual_loss.net.slice3.10.weight', 'loss.perceptual_loss.net.slice3.10.bias', 'loss.perceptual_loss.net.slice3.12.weight', 'loss.perceptual_loss.net.slice3.12.bias', 'loss.perceptual_loss.net.slice3.14.weight', 'loss.perceptual_loss.net.slice3.14.bias', 'loss.perceptual_loss.net.slice4.17.weight', 'loss.perceptual_loss.net.slice4.17.bias', 'loss.perceptual_loss.net.slice4.19.weight', 'loss.perceptual_loss.net.slice4.19.bias', 'loss.perceptual_loss.net.slice4.21.weight', 'loss.perceptual_loss.net.slice4.21.bias', 'loss.perceptual_loss.net.slice5.24.weight', 'loss.perceptual_loss.net.slice5.24.bias', 'loss.perceptual_loss.net.slice5.26.weight', 'loss.perceptual_loss.net.slice5.26.bias', 'loss.perceptual_loss.net.slice5.28.weight', 'loss.perceptual_loss.net.slice5.28.bias', 'loss.perceptual_loss.lin0.model.1.weight', 'loss.perceptual_loss.lin1.model.1.weight', 'loss.perceptual_loss.lin2.model.1.weight', 'loss.perceptual_loss.lin3.model.1.weight', 'loss.perceptual_loss.lin4.model.1.weight', 'loss.flownet.mean', 'loss.flownet.std', 'loss.flownet.basic_module.0.basic_module.0.weight', 'loss.flownet.basic_module.0.basic_module.0.bias', 'loss.flownet.basic_module.0.basic_module.2.weight', 'loss.flownet.basic_module.0.basic_module.2.bias', 'loss.flownet.basic_module.0.basic_module.4.weight', 'loss.flownet.basic_module.0.basic_module.4.bias', 'loss.flownet.basic_module.0.basic_module.6.weight', 'loss.flownet.basic_module.0.basic_module.6.bias', 'loss.flownet.basic_module.0.basic_module.8.weight', 'loss.flownet.basic_module.0.basic_module.8.bias', 'loss.flownet.basic_module.1.basic_module.0.weight', 'loss.flownet.basic_module.1.basic_module.0.bias', 'loss.flownet.basic_module.1.basic_module.2.weight', 'loss.flownet.basic_module.1.basic_module.2.bias', 'loss.flownet.basic_module.1.basic_module.4.weight', 'loss.flownet.basic_module.1.basic_module.4.bias', 'loss.flownet.basic_module.1.basic_module.6.weight', 'loss.flownet.basic_module.1.basic_module.6.bias', 'loss.flownet.basic_module.1.basic_module.8.weight', 'loss.flownet.basic_module.1.basic_module.8.bias', 'loss.flownet.basic_module.2.basic_module.0.weight', 'loss.flownet.basic_module.2.basic_module.0.bias', 'loss.flownet.basic_module.2.basic_module.2.weight', 'loss.flownet.basic_module.2.basic_module.2.bias', 'loss.flownet.basic_module.2.basic_module.4.weight', 'loss.flownet.basic_module.2.basic_module.4.bias', 'loss.flownet.basic_module.2.basic_module.6.weight', 'loss.flownet.basic_module.2.basic_module.6.bias', 'loss.flownet.basic_module.2.basic_module.8.weight', 'loss.flownet.basic_module.2.basic_module.8.bias', 'loss.flownet.basic_module.3.basic_module.0.weight', 'loss.flownet.basic_module.3.basic_module.0.bias', 'loss.flownet.basic_module.3.basic_module.2.weight', 'loss.flownet.basic_module.3.basic_module.2.bias', 'loss.flownet.basic_module.3.basic_module.4.weight', 'loss.flownet.basic_module.3.basic_module.4.bias', 'loss.flownet.basic_module.3.basic_module.6.weight', 'loss.flownet.basic_module.3.basic_module.6.bias', 'loss.flownet.basic_module.3.basic_module.8.weight', 'loss.flownet.basic_module.3.basic_module.8.bias', 'loss.flownet.basic_module.4.basic_module.0.weight', 'loss.flownet.basic_module.4.basic_module.0.bias', 'loss.flownet.basic_module.4.basic_module.2.weight', 'loss.flownet.basic_module.4.basic_module.2.bias', 'loss.flownet.basic_module.4.basic_module.4.weight', 'loss.flownet.basic_module.4.basic_module.4.bias', 'loss.flownet.basic_module.4.basic_module.6.weight', 'loss.flownet.basic_module.4.basic_module.6.bias', 'loss.flownet.basic_module.4.basic_module.8.weight', 'loss.flownet.basic_module.4.basic_module.8.bias', 'loss.flownet.basic_module.5.basic_module.0.weight', 'loss.flownet.basic_module.5.basic_module.0.bias', 'loss.flownet.basic_module.5.basic_module.2.weight', 'loss.flownet.basic_module.5.basic_module.2.bias', 'loss.flownet.basic_module.5.basic_module.4.weight', 'loss.flownet.basic_module.5.basic_module.4.bias', 'loss.flownet.basic_module.5.basic_module.6.weight', 'loss.flownet.basic_module.5.basic_module.6.bias', 'loss.flownet.basic_module.5.basic_module.8.weight', 'loss.flownet.basic_module.5.basic_module.8.bias', 'loss.discriminator.main.0.weight', 'loss.discriminator.main.0.bias', 'loss.discriminator.main.2.weight', 'loss.discriminator.main.3.weight', 'loss.discriminator.main.3.bias', 'loss.discriminator.main.3.running_mean', 'loss.discriminator.main.3.running_var', 'loss.discriminator.main.5.weight', 'loss.discriminator.main.6.weight', 'loss.discriminator.main.6.bias', 'loss.discriminator.main.6.running_mean', 'loss.discriminator.main.6.running_var', 'loss.discriminator.main.8.weight', 'loss.discriminator.main.9.weight', 'loss.discriminator.main.9.bias', 'loss.discriminator.main.9.running_mean', 'loss.discriminator.main.9.running_var', 'loss.discriminator.main.11.weight', 'loss.discriminator.main.11.bias']

meisa233 commented 6 months ago

Which missing parameters it showed?

Encoder Restored from v2-1_512-ema-pruned.ckpt with 230 missing and 994 unexpected keys

Missing Keys: ['decoder.temporal_mixing.temporal_alpha', 'decoder.temporal_mixing.temporal_conv.weight', 'decoder.temporal_mixing.temporal_conv.bias', 'decoder.up.0.temporal_mixing.0.temporal_alpha', 'decoder.up.0.temporal_mixing.0.temporal_conv.weight', 'decoder.up.0.temporal_mixing.0.temporal_conv.bias', 'decoder.up.0.temporal_mixing.1.temporal_alpha', 'decoder.up.0.temporal_mixing.1.temporal_conv.weight', 'decoder.up.0.temporal_mixing.1.temporal_conv.bias', 'decoder.up.0.temporal_mixing.2.temporal_alpha', 'decoder.up.0.temporal_mixing.2.temporal_conv.weight', 'decoder.up.0.temporal_mixing.2.temporal_conv.bias', 'decoder.up.1.temporal_mixing.0.temporal_alpha', 'decoder.up.1.temporal_mixing.0.temporal_conv.weight', 'decoder.up.1.temporal_mixing.0.temporal_conv.bias', 'decoder.up.1.temporal_mixing.1.temporal_alpha', 'decoder.up.1.temporal_mixing.1.temporal_conv.weight', 'decoder.up.1.temporal_mixing.1.temporal_conv.bias', 'decoder.up.1.temporal_mixing.2.temporal_alpha', 'decoder.up.1.temporal_mixing.2.temporal_conv.weight', 'decoder.up.1.temporal_mixing.2.temporal_conv.bias', 'decoder.up.2.temporal_mixing.0.temporal_alpha', 'decoder.up.2.temporal_mixing.0.temporal_conv.weight', 'decoder.up.2.temporal_mixing.0.temporal_conv.bias', 'decoder.up.2.temporal_mixing.1.temporal_alpha', 'decoder.up.2.temporal_mixing.1.temporal_conv.weight', 'decoder.up.2.temporal_mixing.1.temporal_conv.bias', 'decoder.up.2.temporal_mixing.2.temporal_alpha', 'decoder.up.2.temporal_mixing.2.temporal_conv.weight', 'decoder.up.2.temporal_mixing.2.temporal_conv.bias', 'decoder.up.3.temporal_mixing.0.temporal_alpha', 'decoder.up.3.temporal_mixing.0.temporal_conv.weight', 'decoder.up.3.temporal_mixing.0.temporal_conv.bias', 'decoder.up.3.temporal_mixing.1.temporal_alpha', 'decoder.up.3.temporal_mixing.1.temporal_conv.weight', 'decoder.up.3.temporal_mixing.1.temporal_conv.bias', 'decoder.up.3.temporal_mixing.2.temporal_alpha', 'decoder.up.3.temporal_mixing.2.temporal_conv.weight', 'decoder.up.3.temporal_mixing.2.temporal_conv.bias', 'decoder.fusion_layer_2.encode_enc_1.norm1.weight', 'decoder.fusion_layer_2.encode_enc_1.norm1.bias', 'decoder.fusion_layer_2.encode_enc_1.conv1.weight', 'decoder.fusion_layer_2.encode_enc_1.conv1.bias', 'decoder.fusion_layer_2.encode_enc_1.norm2.weight', 'decoder.fusion_layer_2.encode_enc_1.norm2.bias', 'decoder.fusion_layer_2.encode_enc_1.conv2.weight', 'decoder.fusion_layer_2.encode_enc_1.conv2.bias', 'decoder.fusion_layer_2.encode_enc_1.conv_out.weight', 'decoder.fusion_layer_2.encode_enc_1.conv_out.bias', 'decoder.fusion_layer_2.encode_enc_2.0.conv1.weight', 'decoder.fusion_layer_2.encode_enc_2.0.conv1.bias', 'decoder.fusion_layer_2.encode_enc_2.0.conv2.weight', 'decoder.fusion_layer_2.encode_enc_2.0.conv2.bias', 'decoder.fusion_layer_2.encode_enc_2.0.conv3.weight', 'decoder.fusion_layer_2.encode_enc_2.0.conv3.bias', 'decoder.fusion_layer_2.encode_enc_2.0.conv4.weight', 'decoder.fusion_layer_2.encode_enc_2.0.conv4.bias', 'decoder.fusion_layer_2.encode_enc_2.0.conv5.weight', 'decoder.fusion_layer_2.encode_enc_2.0.conv5.bias', 'decoder.fusion_layer_2.encode_enc_2.1.conv1.weight', 'decoder.fusion_layer_2.encode_enc_2.1.conv1.bias', 'decoder.fusion_layer_2.encode_enc_2.1.conv2.weight', 'decoder.fusion_layer_2.encode_enc_2.1.conv2.bias', 'decoder.fusion_layer_2.encode_enc_2.1.conv3.weight', 'decoder.fusion_layer_2.encode_enc_2.1.conv3.bias', 'decoder.fusion_layer_2.encode_enc_2.1.conv4.weight', 'decoder.fusion_layer_2.encode_enc_2.1.conv4.bias', 'decoder.fusion_layer_2.encode_enc_2.1.conv5.weight', 'decoder.fusion_layer_2.encode_enc_2.1.conv5.bias', 'decoder.fusion_layer_2.encode_enc_3.norm1.weight', 'decoder.fusion_layer_2.encode_enc_3.norm1.bias', 'decoder.fusion_layer_2.encode_enc_3.conv1.weight', 'decoder.fusion_layer_2.encode_enc_3.conv1.bias', 'decoder.fusion_layer_2.encode_enc_3.norm2.weight', 'decoder.fusion_layer_2.encode_enc_3.norm2.bias', 'decoder.fusion_layer_2.encode_enc_3.conv2.weight', 'decoder.fusion_layer_2.encode_enc_3.conv2.bias', 'decoder.fusion_layer_1.encode_enc_1.norm1.weight', 'decoder.fusion_layer_1.encode_enc_1.norm1.bias', 'decoder.fusion_layer_1.encode_enc_1.conv1.weight', 'decoder.fusion_layer_1.encode_enc_1.conv1.bias', 'decoder.fusion_layer_1.encode_enc_1.norm2.weight', 'decoder.fusion_layer_1.encode_enc_1.norm2.bias', 'decoder.fusion_layer_1.encode_enc_1.conv2.weight', 'decoder.fusion_layer_1.encode_enc_1.conv2.bias', 'decoder.fusion_layer_1.encode_enc_1.conv_out.weight', 'decoder.fusion_layer_1.encode_enc_1.conv_out.bias', 'decoder.fusion_layer_1.encode_enc_2.0.conv1.weight', 'decoder.fusion_layer_1.encode_enc_2.0.conv1.bias', 'decoder.fusion_layer_1.encode_enc_2.0.conv2.weight', 'decoder.fusion_layer_1.encode_enc_2.0.conv2.bias', 'decoder.fusion_layer_1.encode_enc_2.0.conv3.weight', 'decoder.fusion_layer_1.encode_enc_2.0.conv3.bias', 'decoder.fusion_layer_1.encode_enc_2.0.conv4.weight', 'decoder.fusion_layer_1.encode_enc_2.0.conv4.bias', 'decoder.fusion_layer_1.encode_enc_2.0.conv5.weight', 'decoder.fusion_layer_1.encode_enc_2.0.conv5.bias', 'decoder.fusion_layer_1.encode_enc_2.1.conv1.weight', 'decoder.fusion_layer_1.encode_enc_2.1.conv1.bias', 'decoder.fusion_layer_1.encode_enc_2.1.conv2.weight', 'decoder.fusion_layer_1.encode_enc_2.1.conv2.bias', 'decoder.fusion_layer_1.encode_enc_2.1.conv3.weight', 'decoder.fusion_layer_1.encode_enc_2.1.conv3.bias', 'decoder.fusion_layer_1.encode_enc_2.1.conv4.weight', 'decoder.fusion_layer_1.encode_enc_2.1.conv4.bias', 'decoder.fusion_layer_1.encode_enc_2.1.conv5.weight', 'decoder.fusion_layer_1.encode_enc_2.1.conv5.bias', 'decoder.fusion_layer_1.encode_enc_3.norm1.weight', 'decoder.fusion_layer_1.encode_enc_3.norm1.bias', 'decoder.fusion_layer_1.encode_enc_3.conv1.weight', 'decoder.fusion_layer_1.encode_enc_3.conv1.bias', 'decoder.fusion_layer_1.encode_enc_3.norm2.weight', 'decoder.fusion_layer_1.encode_enc_3.norm2.bias', 'decoder.fusion_layer_1.encode_enc_3.conv2.weight', 'decoder.fusion_layer_1.encode_enc_3.conv2.bias', 'loss.logvar', 'loss.perceptual_loss.scaling_layer.shift', 'loss.perceptual_loss.scaling_layer.scale', 'loss.perceptual_loss.net.slice1.0.weight', 'loss.perceptual_loss.net.slice1.0.bias', 'loss.perceptual_loss.net.slice1.2.weight', 'loss.perceptual_loss.net.slice1.2.bias', 'loss.perceptual_loss.net.slice2.5.weight', 'loss.perceptual_loss.net.slice2.5.bias', 'loss.perceptual_loss.net.slice2.7.weight', 'loss.perceptual_loss.net.slice2.7.bias', 'loss.perceptual_loss.net.slice3.10.weight', 'loss.perceptual_loss.net.slice3.10.bias', 'loss.perceptual_loss.net.slice3.12.weight', 'loss.perceptual_loss.net.slice3.12.bias', 'loss.perceptual_loss.net.slice3.14.weight', 'loss.perceptual_loss.net.slice3.14.bias', 'loss.perceptual_loss.net.slice4.17.weight', 'loss.perceptual_loss.net.slice4.17.bias', 'loss.perceptual_loss.net.slice4.19.weight', 'loss.perceptual_loss.net.slice4.19.bias', 'loss.perceptual_loss.net.slice4.21.weight', 'loss.perceptual_loss.net.slice4.21.bias', 'loss.perceptual_loss.net.slice5.24.weight', 'loss.perceptual_loss.net.slice5.24.bias', 'loss.perceptual_loss.net.slice5.26.weight', 'loss.perceptual_loss.net.slice5.26.bias', 'loss.perceptual_loss.net.slice5.28.weight', 'loss.perceptual_loss.net.slice5.28.bias', 'loss.perceptual_loss.lin0.model.1.weight', 'loss.perceptual_loss.lin1.model.1.weight', 'loss.perceptual_loss.lin2.model.1.weight', 'loss.perceptual_loss.lin3.model.1.weight', 'loss.perceptual_loss.lin4.model.1.weight', 'loss.flownet.mean', 'loss.flownet.std', 'loss.flownet.basic_module.0.basic_module.0.weight', 'loss.flownet.basic_module.0.basic_module.0.bias', 'loss.flownet.basic_module.0.basic_module.2.weight', 'loss.flownet.basic_module.0.basic_module.2.bias', 'loss.flownet.basic_module.0.basic_module.4.weight', 'loss.flownet.basic_module.0.basic_module.4.bias', 'loss.flownet.basic_module.0.basic_module.6.weight', 'loss.flownet.basic_module.0.basic_module.6.bias', 'loss.flownet.basic_module.0.basic_module.8.weight', 'loss.flownet.basic_module.0.basic_module.8.bias', 'loss.flownet.basic_module.1.basic_module.0.weight', 'loss.flownet.basic_module.1.basic_module.0.bias', 'loss.flownet.basic_module.1.basic_module.2.weight', 'loss.flownet.basic_module.1.basic_module.2.bias', 'loss.flownet.basic_module.1.basic_module.4.weight', 'loss.flownet.basic_module.1.basic_module.4.bias', 'loss.flownet.basic_module.1.basic_module.6.weight', 'loss.flownet.basic_module.1.basic_module.6.bias', 'loss.flownet.basic_module.1.basic_module.8.weight', 'loss.flownet.basic_module.1.basic_module.8.bias', 'loss.flownet.basic_module.2.basic_module.0.weight', 'loss.flownet.basic_module.2.basic_module.0.bias', 'loss.flownet.basic_module.2.basic_module.2.weight', 'loss.flownet.basic_module.2.basic_module.2.bias', 'loss.flownet.basic_module.2.basic_module.4.weight', 'loss.flownet.basic_module.2.basic_module.4.bias', 'loss.flownet.basic_module.2.basic_module.6.weight', 'loss.flownet.basic_module.2.basic_module.6.bias', 'loss.flownet.basic_module.2.basic_module.8.weight', 'loss.flownet.basic_module.2.basic_module.8.bias', 'loss.flownet.basic_module.3.basic_module.0.weight', 'loss.flownet.basic_module.3.basic_module.0.bias', 'loss.flownet.basic_module.3.basic_module.2.weight', 'loss.flownet.basic_module.3.basic_module.2.bias', 'loss.flownet.basic_module.3.basic_module.4.weight', 'loss.flownet.basic_module.3.basic_module.4.bias', 'loss.flownet.basic_module.3.basic_module.6.weight', 'loss.flownet.basic_module.3.basic_module.6.bias', 'loss.flownet.basic_module.3.basic_module.8.weight', 'loss.flownet.basic_module.3.basic_module.8.bias', 'loss.flownet.basic_module.4.basic_module.0.weight', 'loss.flownet.basic_module.4.basic_module.0.bias', 'loss.flownet.basic_module.4.basic_module.2.weight', 'loss.flownet.basic_module.4.basic_module.2.bias', 'loss.flownet.basic_module.4.basic_module.4.weight', 'loss.flownet.basic_module.4.basic_module.4.bias', 'loss.flownet.basic_module.4.basic_module.6.weight', 'loss.flownet.basic_module.4.basic_module.6.bias', 'loss.flownet.basic_module.4.basic_module.8.weight', 'loss.flownet.basic_module.4.basic_module.8.bias', 'loss.flownet.basic_module.5.basic_module.0.weight', 'loss.flownet.basic_module.5.basic_module.0.bias', 'loss.flownet.basic_module.5.basic_module.2.weight', 'loss.flownet.basic_module.5.basic_module.2.bias', 'loss.flownet.basic_module.5.basic_module.4.weight', 'loss.flownet.basic_module.5.basic_module.4.bias', 'loss.flownet.basic_module.5.basic_module.6.weight', 'loss.flownet.basic_module.5.basic_module.6.bias', 'loss.flownet.basic_module.5.basic_module.8.weight', 'loss.flownet.basic_module.5.basic_module.8.bias', 'loss.discriminator.main.0.weight', 'loss.discriminator.main.0.bias', 'loss.discriminator.main.2.weight', 'loss.discriminator.main.3.weight', 'loss.discriminator.main.3.bias', 'loss.discriminator.main.3.running_mean', 'loss.discriminator.main.3.running_var', 'loss.discriminator.main.5.weight', 'loss.discriminator.main.6.weight', 'loss.discriminator.main.6.bias', 'loss.discriminator.main.6.running_mean', 'loss.discriminator.main.6.running_var', 'loss.discriminator.main.8.weight', 'loss.discriminator.main.9.weight', 'loss.discriminator.main.9.bias', 'loss.discriminator.main.9.running_mean', 'loss.discriminator.main.9.running_var', 'loss.discriminator.main.11.weight', 'loss.discriminator.main.11.bias']

This is just my personal guess: you may have downloaded the wrong model, or the weights of the model do not match the model file.