Update lm.py to suit OPT-like models

jalammar / ecco

Explain, analyze, and visualize NLP language models. Ecco creates interactive visualizations directly in Jupyter notebooks explaining the behavior of Transformer-based language models (like GPT2, BERT, RoBERTA, T5, and T0).

https://ecco.readthedocs.io

BSD 3-Clause "New" or "Revised" License

1.96k stars 167 forks source link

Update lm.py to suit OPT-like models #99

Closed BiEchi closed 1 year ago

BiEchi commented 1 year ago

Some newer models like OPT treat embedding layers as a separate layer 'Embedding' instead of pure tensor.