[Manual Test] Create and sync a knowledge base from Apache Druid

martyna-mindsdb commented 6 days ago

MindsDB offers the ability to create a Knowledge Base (KB) and synchronize data from any supported data source into it. This issue provides an example of creating a Knowledge Base and automatically syncing content from Apache Druid.

The Knowledge Base will use OpenAI’s embedding model and ChromaDB as the default vector store. The end goal is to showcase an automated process where new data from Apache Druid is continuously added to the Knowledge Base using MindsDB's JOBS feature.

Steps for Contributors :checkered_flag: :shipit:

Create a New Knowledge Base :new: :

Use the CREATE KNOWLEDGE_BASE statement to establish a new KB. See examples of how to create and use a knowledge base.
Integrate Apache Druid :open_file_folder: :

Set up Apache Druid integration to allow data transfer from Apache Druid into the MindsDB environment.
Automate Data Sync using JOBS :timer_clock: :

Create a JOB that automates the process of adding new data from the Apache Druid source to the Knowledge Base.
Testing and Verification :ballot_box_with_check: :

Ensure that the data is correctly synced into the Knowledge Base. If you encounter issues, open a new issue or submit a PR to resolve the problem.

Additional Resources :books:

Deliverables :heavy_check_mark:

An example of a successfully created and synced Knowledge Base with content coming from Apache Druid posted on the issue.
The JOB configuration that ensures automated syncing of data from Apache Druid to the KB.
Verification that the Knowledge Base is functional by querying it for content.
In case of issues, reported bugs or PR that solves them

:loudspeaker: Additional rewards :trophy:

This issue brings :three: :zero: points for SWAG and entry into the draw for a :point_down: :

:three: :x: :computer: Razer Blade 16 Gaming Laptop
:shirt: :bear: MindsDB Swag :information_source: For more info check out https://mindsdb.com/hacktoberfest :point_left:

NOTE: Multiple contributors can work on this issue simultaneously without being assigned. Upon completion, share the screenshots below (if tests succeeded) or create a bug issue (if tests failed).