Closed binary-husky closed 8 months ago
You need to have cargo installed because we did not push wheels for mac with py>3.10. @Narsil should we try go up to 3.11 / 3.12 or are we limited by the runners ?
You need to have cargo installed because we did not push wheels for mac with py>3.10. @Narsil should we try go up to 3.11 / 3.12 or are we limited by the runners ?
No, I using x86 windows os, not mac. Additionally, I still cannot install tokenizers after pip install cargo
. (In python 3.11, everything is fine even without cargo)
Sorry for the confusion. You have to install rust, and probably not through pip. We compiled the wheels for python up to 3.11 not above
@ArthurZucker Would you be up to do a patch release with adding 3.12 support ?
Sure, I'll work on this this week 😉
Hey @ArthurZucker would the lack of wheel compilation also be the reason for the following error?: /lib/python3.11/site-packages/tokenizers/tokenizers.cpython-311-x86_64-linux-gnu.so: ELF load command past end of file
I'm on python 3.11.4.
I can't get chromadb to import tokenizers, it says it doesnt exist even though it does. I'm not sure if I should start a separate issue.
ELF load command past end of file
This means corrupted file, probably unfinished download or something along those lines, delete the folder a install again I think.
See #1406 that will try to release for windows x python 3.12
Christmas delays planning a release for tomorrow 🤗
Another option is to install tokenizers through conda:
conda install conda-forge::tokenizers
Reproducing:
pip install tokenizers
× Encountered error while generating package metadata. ╰─> See above for output.
note: This is an issue with the package mentioned above, not pip. hint: See above for details.