search

HyperGAI / HPT

HPT - Open Multimodal LLMs from HyperGAI
https://www.hypergai.com/
Apache License 2.0
301 stars 14 forks source link

hpt-edge-1-5 use_cache error #11

Closed HongShihan closed 2 weeks ago

HongShihan commented 3 weeks ago

Trying demo.py with hpt-edge-1-5 model I get the following error:

File "/opt/anaconda/envs/kosmos/lib/python3.11/site-packages/transformers/generation/utils.py", line 1322, in _get_initial_cache_position past_length = model_kwargs["past_key_values"][0][0].shape[2]


TypeError: 'NoneType' object is not subscriptable

With some debugging it appears to be caused by "use_cache"  being True in the huggingface function mentioned in the error even though it's False in the model config. The model seems to run correctly if I force it to False (by overwriting the function).
HongShihan commented 2 weeks ago

Probably caused by huggingface issue: https://github.com/huggingface/transformers/issues/31017

Adding check for past_key_values=None resolves the crash.