search

64bit / async-openai

Rust library for OpenAI
https://docs.rs/async-openai
MIT License
1.17k stars 178 forks source link

Tokenizer support #148

Closed Dreaming-Codes closed 6 months ago

Dreaming-Codes commented 11 months ago

Whenever there are updates to this library, it causes compatibility issues with tiktoken-rs. It could be better to flip the script and have this library depend on tiktoken-rs instead, especially since this library often needs changes to keep up with OpenAI API updates.

https://github.com/zurawiki/tiktoken-rs/issues/50

64bit commented 11 months ago

Hey thanks for the issue and description. I dont think this fits into scope of async-openai.

Dreaming-Codes commented 10 months ago

I see your point, but carrying this out seems impractical, considering this library is updated with every OpenAI release. Unless I'm missing something, I think isn't possible to depend on a specific range of library versions.

64bit commented 8 months ago

Sorry I don't have further input on this - you and zurawiki are the right subject matter experts on this.