Think about how to best support batched inferences with e.g. anthropic and OpenAI async APIs - Githubissues

tensorzero / tensorzero

make LLMs improve through experience

https://tensorzero.com

Apache License 2.0

396 stars 17 forks source link

Think about how to best support batched inferences with e.g. anthropic and OpenAI async APIs #379

Closed virajmehta closed 3 weeks ago

GabrielBianconi commented 3 weeks ago

Duplicated: #186