search

ControlNet / MARLIN

[CVPR] MARLIN: Masked Autoencoder for facial video Representation LearnINg
https://openaccess.thecvf.com/content/CVPR2023/html/Cai_MARLIN_Masked_Autoencoder_for_Facial_Video_Representation_LearnINg_CVPR_2023_paper
Other
209 stars 20 forks source link

load the #13

Closed Octopus-Detective closed 3 months ago

Octopus-Detective commented 8 months ago

always fail to load the pretrained model. layers.o.linear.weight False .layers.o.linear.bias False .layers.1.linear.weight False .layers.1.linear.bias False...

ControlNet commented 8 months ago

Could you please provide the minimum example for reproduction? Seems it works fine from my end.

Octopus-Detective commented 8 months ago

I followed your instructions to load the model parameters with the following code:

from marlin_pytorch import Marlin

marlin_model = Marlin.from_file("marlin_vit_base_ytf", "./models/Marlin/marlin_vit_base_ytf.full.pt")

The path "./models/Marlin/marlin_vit_base_ytf.full.pt" is where I put the downloaded parameters from "https://github.com/ControlNet/MARLIN/blob/master/MODEL_ZOO.md". However, after loading, I keep getting messages like:

module.marlin_model.encoder.patch_embedding.projection.weight False
module.marlin_model.encoder.patch_embedding.projection.bias False
module.marlin_model.encoder.pos_embedding.emb False
...

It seems like the parameters were not loaded correctly. Could you please help me understand what might be going wrong?

ControlNet commented 8 months ago

It also works fine from my end. Could you please provide the versions of python, pytorch, marlin_pytorch. image

Maybe it is because the checkpoint is broken when downloading. Can you try re-download the checkpoint?

Octopus-Detective commented 8 months ago

Thank you for your prompt response. The versions of my environment are Python 3.8, PyTorch 2.1, and marlin-pytorch 0.3.4.

Octopus-Detective commented 8 months ago

i have tried to re-download the checkpoint for several times, but it did not work.

ControlNet commented 8 months ago

I tried the exactly same versions and it will work. I'm not sure about the problem. Could you please share the error message?

Octopus-Detective commented 8 months ago

Thank you so much for your help. I'm not experiencing any errors, but it just shows that the parameters can't be loaded! Could you perhaps send a copy of the parameters you're using to my email? My email address is yhzhang@mail.ustc.edu.cn.

ControlNet commented 8 months ago

I think there is no other version of checkpoints for you. Please try the current one in different environments.