search

facebookresearch / textlesslib

Library for Textless Spoken Language Processing
MIT License
518 stars 50 forks source link

[TWIST] Error(s) in loading state_dict for HubertModel #35

Open RoyHirsch opened 5 months ago

RoyHirsch commented 5 months ago

Hi, thank you for the reo! while trying to re-run TWIST we've encounted the following error when initializing CodeHiFiGANVocoder:

Use: dense_model, quantizer_model, vocab = "mhubert-base-25hz", "kmeans", 500

Got the error: RuntimeError: Error(s) in loading state_dict for HubertModel: Missing key(s) in state_dict: "encoder.pos_conv.0.weight_g", "encoder.pos_conv.0.weight_v". Unexpected key(s) in state_dict: "encoder.pos_conv.0.weight", "encoder.pos_conv.0.running_mean", "encoder.pos_conv.0.running_var", "encoder.pos_conv.0.num_batches_tracked", "encoder.pos_conv.1.weight", "encoder.pos_conv.1.bias".

It seems like some of the keys in the state_dict are incompatible, we would appreciate your help.

Roy

jjery2243542 commented 5 months ago

Hi, Thanks for bring out this question and thanks the authors for the great repo! I am having the same issue with the loading of hubert. I assume the model downloaded is not the correct one. Is there any plan to fix it?

Ju-Chieh

xycforgithub commented 2 months ago

Same question here. Hope that the authors can have some way to fix it!

hassidm commented 2 months ago

Hello everyone,

Thank you for reaching out. I attempted to reproduce the error in a new environment, but was unsuccessful - the code worked for me. Could you please try again from a fresh environment?

Thank you.

xycforgithub commented 2 months ago

I resolved the issue by pulling the latest version of fairseq (the textless repo suggests some specific commit, which doesn't work).