search

wagtail / wagtail-ai

Get help with your Wagtail content using AI superpowers.
https://wagtail-ai.readthedocs.io/latest
MIT License
122 stars 20 forks source link

Use wagtail-vector-index as a dependency to reuse AI utils #90

Open tomusher opened 3 weeks ago

tomusher commented 3 weeks ago

wagtail-vector-index has support for additional AI backends like LiteLLM. To stop us having to replicate AI changes between packages, wagtail-ai could depend on wagtail-vector-index and reuse the AI backends from it.

The vector index features are likely to be used in future Wagtail AI features anyway.

tm-kn commented 2 weeks ago

I think another way you can achieve that is to consider the following solutions:

As for the next step, I'd suggest creating a decision record that should explain:

  1. Why the decision was taken in the first place to have them as separate packages?
  2. Why is this a problem?
  3. List all options we are considering, and their pros and cons.
  4. Discuss in a group so that the decision can be made more assuredly, taking all perspectives in.

I think that would be a good step to:

  1. Ensure we make the right decision.
  2. Increase the likelihood that we are not going to be undoing this change in the future because we have a clear thought process documented. And also so that we make the right long-term decision.

The vector index features are likely to be used in future Wagtail AI features anyway.

If that's the case, shouldn't the natural next step be to merge both libraries into one? I would personally be on board with that, with a caveat that it's hard to say given the lack of my background in the decision in the first place. I don't understand the benefit that the separation brings at the moment. Especially given that both libraries are meant to be used with Wagtail only and have a lot of crossover in code.

zerolab commented 2 weeks ago

In my mind the two packages serve different purposes and as such should be distinct. I am not quite clear on how much overlap there is between the (llm) bases for the two is. It sounds to me this is the driver. And if there is enough commonality (other than what deps we have) then imho a utils/base package would be better than a monorepo.

So perhaps let's start by documenting the intersection of the two, the plans for wagtail-ai and what it may use from wagtail-vector-index. That should gives a good starting point for a discussion, and an ADR to follow

tomusher commented 2 weeks ago

For some context; The ai_utils module in wagtail-vector-index is now a more complete abstraction around AI providers like LiteLLM, providing async and streaming interfaces for supported backends. It would be good if these backends/features were available to wagtail-ai, so to prevent us from having to continually copy code across/make changes in both places, finding some way to share code seems valuable.

There are some parts of the wagtail-ai specific AI wrapper that is not currently implemented in wagtail-vector-index, like the image_description API, however these should be trivial to copy to a common codebase.

There is also value in allowing users to provide a single Django setting to configure their AI providers - right now if you use both wagtail-ai and wagtail-vector-index, you need to configure your AI backends in both the WAGTAIL_AI and WAGTAIL_VECTOR_INDEX settings.

At the moment I see wagtail-vector-index as a developer-facing package intended to provide a framework and tools on which a developer can build their own AI integrations, so it seems natural that the APIs for interacting with LLMs are going to evolve faster and be more complete in that package. Developers may also want to use the underlying AI abstractions directly so that they can configure their LLM providers in one place and use a unified API to interact with them.

wagtail-ai is intended as a user-facing, 'plug and play' tool to add AI capabilities to the Wagtail admin. Relying on another package to provide the LLM interactions seems reasonable to me, whether this is wagtail-vector-index or some other shared dependency.

Finally, there are plans for wagtail-ai like #71 and #68 that will likely require vector indexing features from wagtail-vector-index when they are built.

My original thinking around this was to: