search

axinc-ai / ailia-models-cpp

C++ version of ailia models repository
18 stars 2 forks source link

Use EncodeWithSpecialTokens API #71

Closed kyakuno closed 1 month ago

kyakuno commented 2 months ago

ailia Tokenizer 1.3でEncodeはSpecialTokensを符号化しなくなったため、EncodeWithSpecialTokens APIに置き換え。