search

dotnet / machinelearning

ML.NET is an open source and cross-platform machine learning framework for .NET.
https://dot.net/ml
MIT License
9.05k stars 1.88k forks source link

[Tokenizers] How to use SentencePieceBpeTokenizer? #7201

Closed Vanh253333 closed 3 months ago

Vanh253333 commented 3 months ago

I have a sentencepiecebpe.model file. How can I use the SentencePieceBpeTokenizer to load this model file and use it to encode text?