Configure the Service Providers

[saranshisatgit]

We want the system admin to configure the system from the front end where they can create a workspace this workspace will have the configuration of LLM's and Vector DB with embedded configured from the front end.

We need to think how can we separate that like how useanything UI does , it lets you connect with multiple service providers and configure your all the settings around that.

1. Shall we need a separate API that manages all of this?
2. Another part is since most LLM's are exposing their services to API how do we configure or control the most aspects of the settings on the LLM and Vector DB
3. If we do it through other API or backend API what needs to be taken care of

In my opinion, we could test the other backend API where we could manage API and let the user configure through the frontend

[saranshisatgit]

AI based response for more clarity

Issue Description

Feature Request: Frontend Configuration for System Admins

We aim to enable system administrators to configure the system from the frontend, allowing them to create workspaces. Each workspace will include configurations for Large Language Models (LLMs) and Vector Databases (DBs), with embeddings configured through the frontend.

Objectives

1. Frontend Configuration:

   • Provide a UI for system administrators to create and configure workspaces.
   • Each workspace should include settings for LLMs and Vector DBs.

2. Service Integration:

   • Consider how to separate configuration similarly to the "useanything" UI, which allows connecting to multiple service providers and configuring settings for each.

3. API Requirements:

   • Determine if a separate API is needed to manage these configurations.
   • Assess the need for a backend API to handle most settings for LLMs and Vector DBs, given that many LLMs expose their services through APIs.

Considerations

• Configuration Management:

  • Evaluate how to control and configure settings for LLMs and Vector DBs via APIs.
  • Explore options for managing these configurations through a backend API while allowing user interaction via the frontend.

• Testing:

  • Propose testing a backend API that manages the configurations.
  • Ensure that the frontend allows users to configure the settings effectively.

Proposed Solution

1. Frontend UI Development:

   • Design and implement a frontend UI that enables system administrators to create and configure workspaces.
   • Ensure the UI allows for detailed configuration of LLMs and Vector DBs.

2. API Design:

   • Develop a dedicated API to manage the configurations of LLMs and Vector DBs.
   • The API should handle communication between the frontend and the underlying services.

3. Backend Integration:

   • Implement a backend service that interacts with the LLMs and Vector DBs through their respective APIs.
   • The backend should provide endpoints for the frontend to retrieve and update configuration settings.

4. Testing and Validation:

   • Conduct thorough testing of the backend API to ensure it handles configuration management correctly.
   • Validate the frontend UI to ensure it provides a seamless configuration experience for system administrators.

Questions

1. API Management:

   • Should we develop a separate API solely for managing configurations, or integrate this functionality into an existing API?

2. Backend vs. Frontend Configuration:

   • What aspects of the LLM and Vector DB settings should be managed by the backend API, and which should be exposed directly to the frontend for configuration?

3. Service Integration Strategy:

   • How can we best separate and manage configurations for different service providers in a manner similar to "useanything" UI?