Explain, analyze, and visualize NLP language models. Ecco creates interactive visualizations directly in Jupyter notebooks explaining the behavior of Transformer-based language models (like GPT2, BERT, RoBERTA, T5, and T0).
Then using the original HuggingFace library using the same code as in ecco (by literally copying the function from lm.py):
outputs=model.generate(
input_ids=input_ids,
attention_mask=attention_mask,
num_beams=1,
# FIXME: +1 in max_length to account for first start token in decoder, find a better way to do this
max_length=1024,
do_sample=False,
top_p=None,
top_k=None,
temperature=1,
eos_token_id=1, pad_token_id=0,
return_dict_in_generate=True,
output_scores=True)
In both cases, we use the same seed (MKDIDTLISNNAL). But the first method produces this: WSKMLVEEDPGFFERLSQAQKPRALFITCSDSRLVPEQ. And the second produces this: WSKMLVEEDPGFFEKLAQAQKPRFLWIGCSDSRVPAERL.
The two functions should give the same sequence of tokens since we are not sampling. There must be a bug in how lm.generates the tokens iteratively (we know the second sequence is the right one).
Hi, thanks for the great work.
I'm generating text with the lm function without sampling:
Then using the original HuggingFace library using the same code as in ecco (by literally copying the function from lm.py):
In both cases, we use the same seed (
MKDIDTLISNNAL). But the first method produces this:WSKMLVEEDPGFFERLSQAQKPRALFITCSDSRLVPEQ. And the second produces this:WSKMLVEEDPGFFEKLAQAQKPRFLWIGCSDSRVPAERL.The two functions should give the same sequence of tokens since we are not sampling. There must be a bug in how lm.generates the tokens iteratively (we know the second sequence is the right one).