Open NasonZ opened 7 months ago
Downgrading transformers fixed this for me, pip install --upgrade transformers==4.33
Just for any future folks stumbling upon this issue (as did I);
pip install --upgrade transformers==4.33
works as expected but does bring the outdated transformers with it (pretty obvious).
I ended up just upgrading transformers to latest and using the ctransformers package for the AutoModelForCausalLM
and using the transformers package for the AutoTokenizer
env:
transformers ==4.35.2 ctransformers==0.2.27+cu121
gives output:
Seems to be a similar issue to #154.
Any suggestions on how to resolve this would be appreciated.