unexpected keyword argument 'past'

[interaxive]

This project looks great, but I'm struggling to get it working properly. I got it running using gpt2-xl but regardless of what I enter for the starting prompt, I get the following error...

starting prompt: Traceback (most recent call last): File "true_poetry.py", line 526, in (probs, past) = expand_node(prompt, None) File "true_poetry.py", line 380, in expand_node logits, past = model(*inputs, past=past) File "/root/.local/share/virtualenvs/-JyKtHOIQ/lib64/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(input, **kwargs) TypeError: forward() got an unexpected keyword argument 'past'

[interaxive]

I notice this same issue is mentioned in the other thread by @exo-pla-net, the root cause appears to be...

"It is a known issue. Huggingface transformers 4.0 has a breaking change: past becomes past_key_values in GPT-2 models. You can use transformers 3.* to walk around or wait for our fix that is coming soon."

I tried the commit offered by AshBT, as well as numerous variations of replacing "past" with "past_key_values" but this results in the following error...

starting prompt: Traceback (most recent call last): File "true_poetry.py", line 526, in (probs, past) = expand_node(prompt, None) File "true_poetry.py", line 381, in expand_node logits[0][0][50256]=-math.inf # no token TypeError: 'str' object does not support item assignment

This problem has persisted for a while, any help finding a solution without downgrading Huggingface would be much appreciated.

[interaxive]

I solved this issue to allow compatibility with newer 4.x versions of Huggingface Transformers, simply edit line 380 of true_poetry.py as follows:

    logits, past = model(**inputs, past_key_values=past, return_dict=False)

See branch: patch-huggingface4

[summerstay]

Thank you, I have made this change in the code.