search

tensorflow / text

Making text a first-class citizen in TensorFlow.
https://www.tensorflow.org/beta/tutorials/tensorflow_text/intro
Apache License 2.0
1.21k stars 333 forks source link

Recommendation: GemmaTokenizer requires tensorflow-text package (Python 3.12) #1274

Open josnin opened 1 month ago

josnin commented 1 month ago

I would like to recommend adding support for Python 3.12 to the TensorFlow Text library. As Python 3.12 is a relatively new release, it would be beneficial to ensure compatibility with this version.

I am encountering an error when using the GemmaTokenizer. The error message states:

 Exception encountered: GemmaTokenizer requires the tensorflow-text package. Please install with pip install tensorflow-text.
davepickup commented 1 month ago

I would also like to see tensorflow-text support Python 3.12. Tensorflow>=2.16 already supports 3.12 so it seems it would make sense to have the same compatibility in the tensorflow-text package.

nicechester commented 1 month ago

Well, I managed to compile tensorflow-text with python 3.12 on aarch64. You should be able to to that if you followed the instruction on your OS. This is the resulting binary. https://github.com/nicechester/tensorflow-docker/blob/main/wheels/tensorflow_text-2.16.1-cp312-cp312-linux_aarch64.whl