search

rajentrivedi / tokenizer-x

OpenAI token calculator
MIT License
69 stars 5 forks source link

does it support cl100k_base for gpt-4? #6

Closed boynet closed 1 year ago

boynet commented 1 year ago

What happened?

tanks a lot for you work, I wonder is it support cl100k_base for gpt-4?

How to reproduce the bug

tried to look at the code couldn't understand it

Package Version

2

PHP Version

8

Laravel Version

9

Which operating systems does with happen with?

Windows

Notes

No response

rajentrivedi commented 1 year ago

Thanx for reporting the issue.

Currently we are not supporting cl100k_base for gpt-4.

rajentrivedi commented 1 year ago

I would like to add this, but currently I don't have GPT-4 access to test this.

rajentrivedi commented 1 year ago

Currently we are not supporting cl100k_base for gpt-4.