Open LeonardRanaldi opened 1 year ago
"lm_head" is a hardcodeed name of the layer at the end of the model. It likely has a different name in bioGPT
In your local version of the code, try replacing self.model.lm_head
here:
https://github.com/jalammar/ecco/blob/0ef61c51b76c48cd8c21c7807c20510565f307de/src/ecco/lm.py#L399
It seems BioGPT uses the name self.model.output_projection
. Try using it instead of self.model.lm_head
I am testing the BioGPT model with your visualizer. After completing the configuration, I have the following bug: AttributeError: 'BioGptForCausalLM' object has no attribute 'lm_head'