Add all supported model tokenizers for input tokens count

atoma-network / atoma-proxy

Atoma's proxy service repository

Apache License 2.0

0 stars 0 forks source link

Closed jorgeantonio21 closed 2 weeks ago

jorgeantonio21 commented 2 weeks ago

This is required to understand how many tokens do we need to pay for, and current available stacks to use

jorgeantonio21 commented 2 weeks ago

Addressed in #21