Add Initial LiteLLM support - Githubissues

wagtail / wagtail-vector-index

Store Wagtail pages & Django models as embeddings in vector databases

https://wagtail-vector-index.readthedocs.io/en/latest/

MIT License

15 stars 10 forks source link

Add Initial LiteLLM support #64

Closed tomusher closed 2 months ago

tomusher commented 3 months ago

Adds LiteLLM as a supported (default) backend which gives us:

Access to more models out of the box
Async APIs for both chat and embeddings
Support for multi-modal models
Token limits provided for each supported model

This change also:

Changes the structure of messages passed to the backends for more control for the API consumer
API consumer can pass arbitrary parameters to the backend - makes the API less generic but unlocks features from each API backend
Restructures responses from the backends so that multiple choices can be returned when needed.
Adds AIStreamingResponse for handling streams.

This PR will be followed up with a PR to add async/streaming support to the VectorIndex methods.