Paper 'Transformer based Pluralistic Image Completion with Reduced Information Loss' in TPAMI 2024 and 'Reduce Information Loss in Transformers for Pluralistic Image Inpainting' in CVPR2022
MIT License
173
stars
15
forks
source link
Can't load model when I train transformer after training pvqvae #18
I use pvqvae_imagenet.yaml and transformer_imagenet.yaml to train on my own dataset. Training pvqvae is OK, but when I train transformer, I meet a model loading problem:
RuntimeError: Error(s) in loading state_dict for MaskedImageInpaintingTransformer:
Missing key(s) in state_dict: "pos_emb", "content_codec.encoder.pre_layers.0.weight", "content_codec.encoder.pre_layers.0.bias", "content_codec.encoder.res_layers.0.layers.1.weight", "content_codec.encoder.res_layers.0.layers.1.bias", ......
Unexpected key(s) in state_dict: "encoder.pre_layers.0.weight", "encoder.pre_layers.0.bias", "encoder.res_layers.0.layers.1.weight", "encoder.res_layers.0.layers.1.bias", "encoder.res_layers.0.layers.3.weight", "encoder.res_layers.0.layers.3.bias", "encoder.res_layers.1.layers.1.weight", "encoder.res_layers.1.layers.1.bias", "encoder.res_layers.1.layers.3.weight", .....
How to solve it?
You should provid the path of pvqvae model weights in the configure file of transformer, then it will be automatically loaded. You can refer to this line for example:
I use pvqvae_imagenet.yaml and transformer_imagenet.yaml to train on my own dataset. Training pvqvae is OK, but when I train transformer, I meet a model loading problem: RuntimeError: Error(s) in loading state_dict for MaskedImageInpaintingTransformer: Missing key(s) in state_dict: "pos_emb", "content_codec.encoder.pre_layers.0.weight", "content_codec.encoder.pre_layers.0.bias", "content_codec.encoder.res_layers.0.layers.1.weight", "content_codec.encoder.res_layers.0.layers.1.bias", ...... Unexpected key(s) in state_dict: "encoder.pre_layers.0.weight", "encoder.pre_layers.0.bias", "encoder.res_layers.0.layers.1.weight", "encoder.res_layers.0.layers.1.bias", "encoder.res_layers.0.layers.3.weight", "encoder.res_layers.0.layers.3.bias", "encoder.res_layers.1.layers.1.weight", "encoder.res_layers.1.layers.1.bias", "encoder.res_layers.1.layers.3.weight", ..... How to solve it?