huggingface / tokenizers

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
https://huggingface.co/docs/tokenizers
Apache License 2.0
8.69k stars 747 forks source link

Add C++ bindings by mlc-ai to README #1379

Closed ShukantPal closed 7 months ago

ShukantPal commented 8 months ago

This change should make the C++ bindings developed by @mlc-ai more discoverable.