search

JohnGiorgi / DeCLUTR

The corresponding code from our paper "DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations". Do not hesitate to open an issue if you run into any trouble!
https://aclanthology.org/2021.acl-long.72/
Apache License 2.0
378 stars 33 forks source link

Models don't load with allennlp>=1.2.0 #164

Open JohnGiorgi opened 3 years ago

JohnGiorgi commented 3 years ago

The pretrained models do not load properly with allennlp>=1.2.0. The error reported is:

RuntimeError: Error loading state dict for DeCLUTR
    Missing keys: []
    Unexpected keys: ['_text_field_embedder.token_embedder_tokens.transformer_model.roberta.pooler.dense.weight', '_text_field_embedder.token_embedder_tokens.transformer_model.roberta.pooler.dense.bias']

For now, I will constrain the dependency to be "allennlp>=1.1.0, <1.2.0", but it would be great to find another solution (short of re-training the model).

JohnGiorgi commented 3 years ago

This problem is solved by migrating to AllenNLP>=2.0.0. I will close this once I have merged the migration and re-trained the models.

JohnGiorgi commented 3 years ago

Also see: https://github.com/allenai/allennlp/pull/4621#issuecomment-690782222

Which may allow us to avoid re-training