search

mistralai / mistral-common

Apache License 2.0
633 stars 57 forks source link

typo: Correct "nornmalized" to "normalized" in SentencePieceTokenizer #20

Open CharlesCNorton opened 3 months ago

CharlesCNorton commented 3 months ago

Corrected a typo in the SentencePieceTokenizer implementation. The word "nornmalized" was misspelled and has been changed to "normalized" to ensure proper spelling in the assertion message "Message content must be normalized".