[Web] Expose getVocabSize and idToToken to web, bump version to 0.1.3

mlc-ai / tokenizers-cpp

Universal cross-platform tokenizers binding to HF and sentencepiece

Apache License 2.0

211 stars 47 forks source link

Closed CharlieFRuan closed 3 months ago

CharlieFRuan commented 3 months ago

We expose getVocabSize() and IdToToken() to web, subsequently bumping the version to 0.1.3. Tested in web/tests/.