search

EmilyAlsentzer / clinicalBERT

repository for Publicly Available Clinical BERT Embeddings
MIT License
673 stars 135 forks source link

Model not able to initalize weights #30

Closed YojanaGadiya closed 4 years ago

YojanaGadiya commented 4 years ago

Dear all,

I am getting an initalization error about loading the model with BertforTokenClassification. Is there any way to get rid of this error?

image

Thank You.

EmilyAlsentzer commented 4 years ago

Can you provide a bit more detail regarding what produced this message? Note that the clinicalBERT model on the Huggingface hub is not fine tuned for NER. Does loading the model as described in issue #29 work?

YojanaGadiya commented 4 years ago

Hey, yes I have used the model for NER using AutomodelforTokencalssification form transformers library. But I get this error.

image

I am not sure whether it affects the pre-model or not though. But in general, I would like to know whether this is normal behavior? if not, how to solve this issue?

EmilyAlsentzer commented 4 years ago

This behavior is expected because the clinical BERT model on the huggingface repo was only trained on the MLM and next sentence prediction tasks. If you want to use the model for NER, you need to fine-tune the model on your own data.