Investigate the "some weights were not initialized" warning on HF models

i-machine-think / diagNNose

diagNNose is a Python library that facilitates a broad set of tools for analysing hidden activations of neural models.

https://diagnnose.readthedocs.io

MIT License

81 stars 8 forks source link

Investigate the "some weights were not initialized" warning on HF models #61

Open jumelet opened 4 years ago

jumelet commented 4 years ago


Some weights of BertForMaskedLM were not initialized from the model checkpoint at bert-base-cased and are newly initialized: ['cls.predictions.decoder.bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.```