Open herbert-zhou opened 7 months ago
And I tried both installing ecco with pip and conda, but I don't think it would make any difference.
The problem is simple: I ran the following lines:
import ecco
lm = ecco.from_pretrained('distilgpt2')
text= "The keys to the cabinet"
output = lm.generate(text, generate=1, do_sample=False)
And I checked the states:
print(output.decoder_hidden_states)
, which prints None
And the error came in when I tried to call any function that involves the decoder hidden state:
output.rankings()
Has anyone been able to fix this error yet?
The issue is that you are not telling the model to track the hidden layers. All you have to do is add "output_hidden_states=True" when you call the generate method. Here is the example:
output = lm.generate(text, generate=1, do_sample=False, output_hidden_states=True)
Hope that helps :)
I installed ecco with python 3.8 and 3.9 (I tried both) on Jan 31, 2024. However, the examples from the notebooks are not working with the common error of "decoder hidden states not found". Other functionalities worked, it is just with all the functions that requires the decoder hidden state (e.g. token rankings).