using cross encoder

[ReySadeghi]

Hi, I'm trying to use cross encoder and I used a fine-tuned model (BERT-base) as a pretrained model, but I got this error:

ValueError: Unrecognized model Should have a model_type key in its config.json, or contain one of the following strings in its name: retribert, t5, mobilebert, distilbert, albert, bert-generation, camembert, xlm-roberta, pegasus, marian, mbart, bart, blenderbot, reformer, longformer, roberta, deberta, flaubert, fsmt, squeezebert, bert, openai-gpt, gpt2, transfo-xl, xlnet, xlm-prophetnet, prophetnet, xlm, ctrl, electra, encoder-decoder, funnel, lxmert, dpr, layoutlm, rag

[nreimers]

Hi @ReySadeghi Old models from huggingface have an outdated config.json file and are not compatible with AutoModel, which is used by the cross-encoder.

Download the model, open the config.json and add the "model_type" key to it. Then you can load it with the CrossEncoder class.

Here you see the config.json of bert-base-uncased: https://huggingface.co/bert-base-uncased/blob/main/config.json

[ReySadeghi]

I loaded the model that was fine-tuned with sentence-transformer and also it has "model_type="bert"" in config.json.

what should be the problem?

[nreimers]

Note, the transformer model is stored in the 0_Transformer folder. When used with a cross-encoder, you have to load that folder, not the parent folder (which has a config.json file specific to sentence transformer).

[ReySadeghi]

oh, yeah, thanks.