search

alasdairforsythe / tokenmonster

Ungreedy subword tokenizer and vocabulary trainer for Python, Go & Javascript
MIT License
548 stars 19 forks source link

charset bug fix #12

Closed codinglover0111 closed 1 year ago

codinglover0111 commented 1 year ago

charset() bug fix if use 'self.charset' it return pointer so I change to 'self.charset()'

alasdairforsythe commented 1 year ago

Thanks it's been fixed. To be sure please update it from pip and also delete the "tokenmonsterserver" file from ~/_tokenmonster