search

sashirestela / simple-openai

A Java library to use the OpenAI Api in the simplest possible way.
MIT License
169 stars 16 forks source link

Do you implement a way of tokenizing text? #158

Closed monkiki closed 4 months ago

monkiki commented 4 months ago

I know the tiktoken Python library, but and even have found Java implementations like jtokkit. Do you include a tokenizer o plan to add one?

Thanks in advance.

sashirestela commented 4 months ago

Hi @monkiki , thanks for using simple-openai. No, I don't and no plan to add any. I think that Jtokkit is a great option.

monkiki commented 4 months ago

Ok, thanks for the info!