search

NKI-AI / STAPLER

STAPLER (Shared TCR And Peptide Language bidirectional Encoder Representations from transformers) is a language model that uses a joint TCRab-peptide input to predict TCRab-peptide specificity.
Apache License 2.0
25 stars 2 forks source link

Missing key(s) in state_dict #3

Open LilySnow opened 2 days ago

LilySnow commented 2 days ago

I tried to run python test.py in tools, but got the following error message. The checkpoint file is downloaded from https://files.aiforoncology.nl/stapler/model/pretrained_model/


Traceback (most recent call last):
  File "/home/lixue1/tools/miniconda3/lib/python3.10/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 92, in _call_target
    return _target_(*args, **kwargs)
  File "/gpfs/home6/lixue1/tcrspec/STAPLER/tools/../stapler/models/stapler_transformer.py", line 35, in __init__
    self.load_model(checkpoint_path)
  File "/gpfs/home6/lixue1/tcrspec/STAPLER/tools/../stapler/models/stapler_transformer.py", line 90, in load_model
    self.load_state_dict(state_dict)
  File "/home/lixue1/tools/miniconda3/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2041, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for STAPLERTransformer:
    Missing key(s) in state_dict: "token_emb.emb.weight", "attn_layers.layers.1.1.ff.0.proj.weight", "attn_layers.layers.1.1.ff.0.proj.bias", "attn_layers.layers.1.1.ff.2.weight", "attn_layers.layers.1.1.ff.2.bias", "attn_layers.layers.3.1.ff.0.proj.weight", "attn_layers.layers.3.1.ff.0.proj.bias", "attn_layers.layers.3.1.ff.2.weight", "attn_layers.layers.3.1.ff.2.bias", "attn_layers.layers.5.1.ff.0.proj.weight", "attn_layers.layers.5.1.ff.0.proj.bias", "attn_layers.layers.5.1.ff.2.weight", "attn_layers.layers.5.1.ff.2.bias", "attn_layers.layers.7.1.ff.0.proj.weight", "attn_layers.layers.7.1.ff.0.proj.bias", "attn_layers.layers.7.1.ff.2.weight", "attn_layers.layers.7.1.ff.2.bias", "attn_layers.layers.9.1.ff.0.proj.weight", "attn_layers.layers.9.1.ff.0.proj.bias", "attn_layers.layers.9.1.ff.2.weight", "attn_layers.layers.9.1.ff.2.bias", "attn_layers.layers.11.1.ff.0.proj.weight", "attn_layers.layers.11.1.ff.0.proj.bias", "attn_layers.layers.11.1.ff.2.weight", "attn_layers.layers.11.1.ff.2.bias", "attn_layers.layers.13.1.ff.0.proj.weight", "attn_layers.layers.13.1.ff.0.proj.bias", "attn_layers.layers.13.1.ff.2.weight", "attn_layers.layers.13.1.ff.2.bias", "attn_layers.layers.15.1.ff.0.proj.weight", "attn_layers.layers.15.1.ff.0.proj.bias", "attn_layers.layers.15.1.ff.2.weight", "attn_layers.layers.15.1.ff.2.bias", "attn_layers.final_norm.weight", "attn_layers.final_norm.bias".
    Unexpected key(s) in state_dict: "norm.weight", "norm.bias", "token_emb.weight", "attn_layers.layers.0.1.to_out.bias", "attn_layers.layers.1.1.net.0.proj.weight", "attn_layers.layers.1.1.net.0.proj.bias", "attn_layers.layers.1.1.net.3.weight", "attn_layers.layers.1.1.net.3.bias", "attn_layers.layers.2.1.to_out.bias", "attn_layers.layers.3.1.net.0.proj.weight", "attn_layers.layers.3.1.net.0.proj.bias", "attn_layers.layers.3.1.net.3.weight", "attn_layers.layers.3.1.net.3.bias", "attn_layers.layers.4.1.to_out.bias", "attn_layers.layers.5.1.net.0.proj.weight", "attn_layers.layers.5.1.net.0.proj.bias", "attn_layers.layers.5.1.net.3.weight", "attn_layers.layers.5.1.net.3.bias", "attn_layers.layers.6.1.to_out.bias", "attn_layers.layers.7.1.net.0.proj.weight", "attn_layers.layers.7.1.net.0.proj.bias", "attn_layers.layers.7.1.net.3.weight", "attn_layers.layers.7.1.net.3.bias", "attn_layers.layers.8.1.to_out.bias", "attn_layers.layers.9.1.net.0.proj.weight", "attn_layers.layers.9.1.net.0.proj.bias", "attn_layers.layers.9.1.net.3.weight", "attn_layers.layers.9.1.net.3.bias", "attn_layers.layers.10.1.to_out.bias", "attn_layers.layers.11.1.net.0.proj.weight", "attn_layers.layers.11.1.net.0.proj.bias", "attn_layers.layers.11.1.net.3.weight", "attn_layers.layers.11.1.net.3.bias", "attn_layers.layers.12.1.to_out.bias", "attn_layers.layers.13.1.net.0.proj.weight", "attn_layers.layers.13.1.net.0.proj.bias", "attn_layers.layers.13.1.net.3.weight", "attn_layers.layers.13.1.net.3.bias", "attn_layers.layers.14.1.to_out.bias", "attn_layers.layers.15.1.net.0.proj.weight", "attn_layers.layers.15.1.net.0.proj.bias", "attn_layers.layers.15.1.net.3.weight", "attn_layers.layers.15.1.net.3.bias".
bpkwee commented 2 days ago

Hi Lily,

Could it be related to this issue? https://github.com/NKI-AI/STAPLER/issues/1

LilySnow commented 2 days ago

The problem is the checkpoint file downloaded from https://files.aiforoncology.nl/stapler/model/pretrained_model/. When I changed it to one of the finetuned model checkpoint file, it works.

bpkwee commented 2 days ago

Your error looks very similar to the last 2 messages in that issue. Does this solve the problem you encounter? https://github.com/NKI-AI/STAPLER/issues/1#issuecomment-1585742847