search

defenseunicorns / leapfrogai

Production-ready Generative AI for local, cloud native, airgap, and edge deployments.
https://leapfrog.ai
Apache License 2.0
257 stars 28 forks source link

chore(api): Do not block when creating/updating VectorStores or Assistants #803

Closed CollectiveUnicorn closed 1 month ago

CollectiveUnicorn commented 3 months ago

Describe what should be investigated or refactored

When hitting the endpoint to create or update VectorStores/Assistants there is a blocking operation that occurs with the indexing service. The implementation needs to be updated so that the indexing is run as a background job so that the user is not blocked waiting for the indexing to finish before being able to use an assistant.

Additional context

FastAPI background tasks are one way to do this. Indexing status is required before this can be done, as otherwise the frontend will be unable to determine when indexing has completed.

CollectiveUnicorn commented 3 months ago

Blocked by https://github.com/defenseunicorns/leapfrogai/issues/788

YrrepNoj commented 2 months ago

885 might mitigate some of this issue.

It appears like our API and EmbeddingsModel was in a fail loop which was causing a single request to repeat multiple times. After #885 the core issue described by this issue might be mitigated.

That said, I believe this issue should stay open. #885 might make this less painful, but it is still a painful thing that we should address.

CollectiveUnicorn commented 2 months ago

Confirming that this still seems to be an issue even after #885 has been merged into main unfortunately