search

Unstructured-IO / unstructured

Open source libraries and APIs to build custom preprocessing pipelines for labeling, training, or production machine learning pipelines.
https://www.unstructured.io/
Apache License 2.0
7.44k stars 580 forks source link

KDB.AI as destination connector #3223

Open alexgiannak opened 2 weeks ago

alexgiannak commented 2 weeks ago

To connect with kdbai cloud please follow the below link https://code.kx.com/kdbai/gettingStarted/kdb-ai-cloud-setup.html

import kdbai_client as kdbai

ftkey = "your-api-key"
session = kdbai.Session(endpoint='your-endpoint',api_key=ftkey)

schema = {'columns': [
         {'name': 'id', 'pytype': 'str'},
         {'name': 'document', 'pytype': 'str'},
         {'name': 'metadata', 'pytype': 'dict'},
         {'name': 'embedding',
             'vectorIndex': {'dims': 384, 'type': 'hnsw', 'metric': 'L2', 'efConstruction': 8, 'M': 8}}]}

session.create_table("elements",schema)
rbiseck3 commented 2 weeks ago

Is there a docker version of the database that can be used to validate the connector locally without needing a cloud connection?