Getting some weights not used warning

[wailoktam]

Hi, I get the following warning when loading a checkpoint trained using your pretrained model. Is there anything wrong with it?

Some weights of the model checkpoint at /content/drive/My Drive/pretrainedBertJa/bert-base-japanese-whole-word-masking were not used when initializing BertForSequenceClassification: ['cls.predictions.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.decoder.weight', 'cls.seq_relationship.weight', 'cls.seq_relationship.bias']

• This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPretraining model).
• This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). Some weights of BertForSequenceClassification were not initialized from the model checkpoint at /content/drive/My Drive/pretrainedBertJa/bert-base-japanese-whole-word-masking and are newly initialized: ['classifier.weight', 'classifier.bias'] You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

[singletongue]

This IS an expected behavior since you are initializing a BertForSequenceClassification model from our BertForPretraining model. See the warning message in the first bullet point.