Closed Kevin-XiongC closed 4 months ago
I think there is a bug in latest transformers implementation
Could you please try downgrading transformers (eg to version 4.37) and retry? It would be extremely helpful!
I think there is a bug in latest transformers implementation
Could you please try downgrading transformers (eg to version 4.37) and retry? It would be extremely helpful!
Yes, after downgrading to 4.37. It works as expected. Thx.
[ENV] transformers 4.39.3 pytorch 2.1.2 cuda=11.8
Thanks for the work! I've try to use PLD in my deployment of LLama7B. However, I find that using argmax(temperture==0) leads to inconsistent decode results compare to output without PLD.![image](https://github.com/apoorvumang/prompt-lookup-decoding/assets/22584786/a130efb1-6aa9-4084-be31-a9c01cc193e7)
Is the implementation on HF correct?