huggingface / tokenizers

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
https://huggingface.co/docs/tokenizers
Apache License 2.0
8.93k stars 779 forks source link

Release Candidate #1388

Closed ArthurZucker closed 10 months ago

HuggingFaceDocBuilderDev commented 10 months ago

The documentation is not available anymore as the PR was closed or merged.