jalammar / ecco

Explain, analyze, and visualize NLP language models. Ecco creates interactive visualizations directly in Jupyter notebooks explaining the behavior of Transformer-based language models (like GPT2, BERT, RoBERTA, T5, and T0).
https://ecco.readthedocs.io
BSD 3-Clause "New" or "Revised" License
1.96k stars 167 forks source link

update requirements and add float16 support #110

Closed SumanthRH closed 7 months ago

SumanthRH commented 7 months ago

What does this PR do?

I've been playing around with the library recently and noticed that some of the files were a bit outdated. This is a simple PR to

Ex: lm = ecco.from_pretrained("distilgpt2", activations=True, torch_dtype=torch.bfloat16)

I've tested attributions and neuron factors notebooks with the new changes and things seem to work well

SumanthRH commented 7 months ago

Okay so I ran some more tests on using bfloat16 and it works for primary attributions, neuron factors, output token scores, token rankings and for visualizing the evolution of a token. Hoping to see this merged @jalammar !

jalammar commented 7 months ago

Thanks @SumanthRH! Merging this into a branch to update the CI/CD python versions.

Which version of python did you test this on?

SumanthRH commented 7 months ago

I tested this on python 3.10!