search

pkoukk / tiktoken-go

go version of tiktoken
MIT License
649 stars 71 forks source link

Is there any plan to adapt the llama model? #26

Open maplessssy opened 1 year ago

pkoukk commented 1 year ago

I didn't find that model in open-ai's website, please give me more info.

maplessssy commented 1 year ago

llama is an open source model of facebook.

FYI: js version is here: https://github.com/belladoreai/llama-tokenizer-js

pkoukk commented 1 year ago

sorry, this project is just for open-ai, so I have no plan about that recently.