search

jalammar / ecco

Explain, analyze, and visualize NLP language models. Ecco creates interactive visualizations directly in Jupyter notebooks explaining the behavior of Transformer-based language models (like GPT2, BERT, RoBERTA, T5, and T0).
https://ecco.readthedocs.io
BSD 3-Clause "New" or "Revised" License
1.96k stars 167 forks source link

Decoder hidden states not found #113

Open herbert-zhou opened 7 months ago

herbert-zhou commented 7 months ago

I installed ecco with python 3.8 and 3.9 (I tried both) on Jan 31, 2024. However, the examples from the notebooks are not working with the common error of "decoder hidden states not found". Other functionalities worked, it is just with all the functions that requires the decoder hidden state (e.g. token rankings).

herbert-zhou commented 7 months ago

And I tried both installing ecco with pip and conda, but I don't think it would make any difference.

The problem is simple: I ran the following lines:

import ecco lm = ecco.from_pretrained('distilgpt2') text= "The keys to the cabinet" output = lm.generate(text, generate=1, do_sample=False)

And I checked the states: print(output.decoder_hidden_states), which prints None

And the error came in when I tried to call any function that involves the decoder hidden state: output.rankings()

TenshirenDNTU commented 4 months ago

Has anyone been able to fix this error yet?

sachaRfd commented 2 months ago

The issue is that you are not telling the model to track the hidden layers. All you have to do is add "output_hidden_states=True" when you call the generate method. Here is the example:

output = lm.generate(text, generate=1, do_sample=False, output_hidden_states=True)

Hope that helps :)