mindsdb / mindsdb

The platform for building AI from enterprise data
https://mindsdb.com
Other
26.72k stars 4.87k forks source link

[Manual Test] Create and sync a knowledge base from Couchbase #10000

Closed martyna-mindsdb closed 3 days ago

martyna-mindsdb commented 1 week ago

MindsDB offers the ability to create a Knowledge Base (KB) and synchronize data from any supported data source into it. This issue provides an example of creating a Knowledge Base and automatically syncing content from Couchbase.

The Knowledge Base will use OpenAI’s embedding model and ChromaDB as the default vector store. The end goal is to showcase an automated process where new data from Couchbase is continuously added to the Knowledge Base using MindsDB's JOBS feature.

Steps for Contributors :checkered_flag: :shipit:

  1. Create a New Knowledge Base :new: :

    Use the CREATE KNOWLEDGE_BASE statement to establish a new KB. See examples of how to create and use a knowledge base.

  2. Integrate Couchbase :open_file_folder: :

    Set up Couchbase integration to allow data transfer from Couchbase into the MindsDB environment.

  3. Automate Data Sync using JOBS :timer_clock: :

    Create a JOB that automates the process of adding new data from the Couchbase source to the Knowledge Base.

  4. Testing and Verification :ballot_box_with_check: :

    Ensure that the data is correctly synced into the Knowledge Base. If you encounter issues, open a new issue or submit a PR to resolve the problem.

Additional Resources :books:

Deliverables :heavy_check_mark:

:loudspeaker: Additional rewards :trophy:

This issue brings :six: :zero: points for SWAG and entry into the draw for a :point_down: :


NOTE: Multiple contributors can work on this issue simultaneously without being assigned. Upon completion, share the screenshots below (if tests succeeded) or create a bug issue (if tests failed).

ritwickrajmakhal commented 1 week ago

Works fine ✅

Screencast from 2024-10-23 22-50-03.webm

Better-Boy commented 1 week ago

Knowledge base with couchbase is working fine. FYI - hugging face embedding is used here.

https://github.com/user-attachments/assets/894622fd-aa3c-4f60-a38c-2a56c046c296