allenai / scibert_scivocab_uncased instead of allenai/scifact_scivocab_uncased

[antoniakrm]

Hello,

I am trying to train your model from scratch following the instructions in training.md. For the model parameter you mention:

--model: Model argument allows you to select what pretrained model to use. Options include roberta-base, roberta-large, allenai/scifact_scivocab_uncased, allenai/biomed_roberta_base, and PATH TO CHECKPOINT. Note that if you select non-roberta model. You may need to edit the script where it is commented.

In order to train the scibert model I have changed {'params': model.roberta.parameters(), 'lr': args.lr_base}, to {'params': model.bert.parameters(), 'lr': args.lr_base}. However, I get the following traceback:

OSError: Can't load 'allenai/scifact_scivocab_uncased'. Make sure that:

• 'allenai/scifact_scivocab_uncased' is a correct model identifier listed on 'https://huggingface.co/models'

• or 'allenai/scifact_scivocab_uncased' is the correct path to a directory containing a 'config.json' file

I cannot find 'allenai/scifact_scivocab_uncased' in huging face, and the code runs when using 'allenai/scibert_scivocab_uncased' instead.

Thank you very much for your help and time. I really appreciate your work! Yours Sincerely, Antonia

[dwadden]

Hi Antonia,

Thanks, you've caught a typo! It should say allenai/scibert_scivocab_uncased. I've updated the docs accordingly. Thanks for pointing this out.

Dave