search

microsoft / ProphetNet

A research project for natural language generation, containing the official implementations by MSRA NLC team.
MIT License
686 stars 109 forks source link

Unable to load the GENIE model #64

Open CSerxy opened 1 year ago

CSerxy commented 1 year ago

Hi GENIE authors @lzh0525 @qiweizhen ,

Thanks for your excellent work. However, I found it hard to load your models. When I tried to do the inference on XSum or cnndm from your fine-tuned models. I got below errors:

RuntimeError: Error(s) in loading state_dict for CrossAttention_Diffusion_LM: Unexpected key(s) in state_dict: "passage_encoder.embeddings.position_ids".

from Line model.load_state_dict(model_saved_state.model_dict) in ProphetNet/GENIE/Genie_Generate.py file.

I wonder if this relates to the PyTorch version I am using. Since I was able to load your model months ago, but cannot do it now. I kind of forgetting the version I used and didn't succeed to get the correct ones.

Also, would it be better to clarify upload your version in a requirement.txt file?

Many thanks!

DinDjarin666 commented 5 months ago

I met the same problem. Have you solved it yet?

SebOchs commented 3 months ago

I found a difference in the saved model and the original model's state dict. My work-around was replacing l. 219 in Genie_Generate.py: model.load_state_dict(model_saved_state.model_dict) with model_saved_state.model_dict["passage_encoder.embeddings.position_ids"] = model_saved_state.model_dict["position_ids"] model.load_state_dict(model_saved_state.model_dict)

MidiyaZhu commented 1 week ago

I changed the transformer version into 4.25.1, huggingface-hub=0.25.0 and torch==2.1.0+cu118, the issues solved