Tokenizer issue with Transformers 4.33.0

mit-han-lab / streaming-llm

[ICLR 2024] Efficient Streaming Language Models with Attention Sinks

https://arxiv.org/abs/2309.17453

MIT License

6.59k stars 361 forks source link

Open PedemonteGiacomo opened 3 months ago

PedemonteGiacomo commented 3 months ago

Have anyone encountered the error relevant to the building wheel of tokenizers while performing pip install transformers==4.33.0