search

tensorflow / text

Making text a first-class citizen in TensorFlow.
https://www.tensorflow.org/beta/tutorials/tensorflow_text/intro
Apache License 2.0
1.23k stars 344 forks source link

Support BYTE piece type in FastSentencepieceTokenizer #1250

Closed copybara-service[bot] closed 9 months ago

copybara-service[bot] commented 9 months ago

Support BYTE piece type in FastSentencepieceTokenizer