Unexpected key(s) in state_dict: "embeddings.position_ids"

chao1224 / MoleculeSTM

Multi-modal Molecule Structure-text Model for Text-based Editing and Retrieval, Nat Mach Intell 2023 (https://www.nature.com/articles/s42256-023-00759-6)

https://chao1224.github.io/MoleculeSTM

Other

188 stars 18 forks source link

Unexpected key(s) in state_dict: "embeddings.position_ids" #10

Closed ZwormZ closed 4 months ago

ZwormZ commented 7 months ago

Hello, I have downloaded the toy checkpoint that you provided. However, when I try to run the script "downstream_01_retrieval_ATC_Retrieval.py", I have noticed that the text model dict does not match with "scibert_scivocab_uncased". Is the text model slightly different from Scibert?

chao1224 commented 7 months ago

Hi @ZwormZ , that's interesting. I haven't seen this before, and I'm pretty sure that we haven't changed the scibert model. Can you give the exact CMD and complete screenshot?

ZwormZ commented 7 months ago

Hi，I have printed the Scibert module layer and the MoleculeSTM's text_model name and discovered that the text_model has an additional layer called "embedding.position_ids".

And this is the script

chao1224 commented 6 months ago

Hi @ZwormZ,

I just tried this, and everything works well on my end. I suspect that this might be the HF version issue. BTW. I am using the following:

transformers          4.26.1