bug -object has no attribute 'lm_head'-

jalammar / ecco

Explain, analyze, and visualize NLP language models. Ecco creates interactive visualizations directly in Jupyter notebooks explaining the behavior of Transformer-based language models (like GPT2, BERT, RoBERTA, T5, and T0).

https://ecco.readthedocs.io

BSD 3-Clause "New" or "Revised" License

1.96k stars 167 forks source link

bug -object has no attribute 'lm_head'- #91

Open LeonardRanaldi opened 1 year ago

LeonardRanaldi commented 1 year ago

I am testing the BioGPT model with your visualizer. After completing the configuration, I have the following bug: AttributeError: 'BioGptForCausalLM' object has no attribute 'lm_head'

jalammar commented 1 year ago

"lm_head" is a hardcodeed name of the layer at the end of the model. It likely has a different name in bioGPT

jalammar commented 1 year ago

In your local version of the code, try replacing self.model.lm_head here: https://github.com/jalammar/ecco/blob/0ef61c51b76c48cd8c21c7807c20510565f307de/src/ecco/lm.py#L399

It seems BioGPT uses the name self.model.output_projection. Try using it instead of self.model.lm_head