Closed arthurmarcal closed 4 years ago
Hi @arthurmarcal ,
1) Yes. The first warning is expected because the checkpoint is of a BertForPreTraining
model, which has classification heads for the pretraining tasks (cls
module). The second warning (Some weights of BertForSequenceClassification were not initialized from the model checkpoint...
) is also expected: classifier
is the new classification layer that will be trained on your task.
2) Yes, because this way you only have to learn a few thousand parameters for your task (weights of classifier
module). Otherwise you will have to train all 110M~330M parameters from scratch on your classification dataset.
Cool! Thanks Fabio.
@arthurmarcal , were you able to get this running? If so, did it improve your results or processing time?
transformers==3.0.2 on Colab enviroment
When I load the model into the BertForSequenceClassification class.
I got the following warning msg: Some weights of the model checkpoint at neuralmind/bert-base-portuguese-cased were not used when initializing BertForSequenceClassification
1) Is it OK to use this model for classification tasks? 2) I mean, can I benefit from the pre-trained weights?