search

ControlNet / MARLIN

[CVPR] MARLIN: Masked Autoencoder for facial video Representation LearnINg
https://openaccess.thecvf.com/content/CVPR2023/html/Cai_MARLIN_Masked_Autoencoder_for_Facial_Video_Representation_LearnINg_CVPR_2023_paper
Other
231 stars 20 forks source link

Poor performance of marlin_vit_small_ytf #7

Closed tvaranka closed 1 year ago

tvaranka commented 1 year ago

Hey, thanks for the great work! I have been using the marlin_vit_base_ytf with success when fine-tuning on my custom task. I wanted to try the smaller version marlin_vit_small_ytf due to small dataset. However, I find the performance is extremely poor.

With the base model after just 5 epochs I am able to get around 0.2 accuracy, but with the small model after 100 epochs the accuracy is just 0.02.

It almost seems like using a model with random initialization and that the small model hasn't been pre-trained. Would it be possible that you accidentally uploaded the wrong model?

tvaranka commented 1 year ago

Nevermind, the model is correct, the issue was on my end.

I was using my local machine for the small model with a small batch size, which seems to have caused the issue.

ControlNet commented 1 year ago

Thanks for sharing.