[Question]: fast-tokenizer 不支持python3.11？

PaddlePaddle / PaddleNLP

👑 Easy-to-use and powerful NLP and LLM library with 🤗 Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including 🗂Text Classification, 🔍 Neural Search, ❓ Question Answering, ℹ️ Information Extraction, 📄 Document Intelligence, 💌 Sentiment Analysis etc.

https://paddlenlp.readthedocs.io

Apache License 2.0

11.73k stars 2.87k forks source link

[Question]: fast-tokenizer 不支持python3.11？ #6530

Open 526872163 opened 11 months ago

526872163 commented 11 months ago

请提出你的问题

我使用paddlenlp需要使用python3.11加速项目，但是安装的时候找不到支持python3.11 的fast-tokenizer pip安装包，请尽快提供，此外是否可以使用手动编译源码的方式支持python3.11

w5688414 commented 2 months ago

目前由于人力问题，已经停止更新fast-tokenizer了，欢迎开发者贡献。

https://github.com/PaddlePaddle/PaddleNLP/tree/develop/fast_tokenizer