search

pkoukk / tiktoken-go

go version of tiktoken
MIT License
601 stars 67 forks source link

support all chat model from openai #24

Closed vaayne closed 1 year ago

pkoukk commented 1 year ago

These models are coverd by MODEL_PREFIX_TO_ENCODING.