[Feature]: .patch pattern & API - similar to what Instructor does

BerriAI / litellm

Call all LLM APIs using the OpenAI format. Use Bedrock, Azure, OpenAI, Cohere, Anthropic, Ollama, Sagemaker, HuggingFace, Replicate (100+ LLMs)

https://docs.litellm.ai/docs/

Other

10.11k stars 1.13k forks source link

[Feature]: .patch pattern & API - similar to what Instructor does #4367

Open ChristianWeyer opened 6 days ago

ChristianWeyer commented 6 days ago

The Feature

It would be very feasible to have a client.patch(...) method which gives us an OpenAI SDK client instance with the LiteLLM features enabled. Similar to what Instructor does.

Ideally, LiteLLM and Instructor would still be able to seamlessly work together.

Motivation, pitch

Ease of use, homogenized programming model. Interoperability with other libs (which accept OpenAI clients).

Twitter / LinkedIn details

https://x.com/christianweyer