tursodatabase / libsql

libSQL is a fork of SQLite that is both Open Source, and Open Contributions.
https://turso.tech/libsql
MIT License
9.54k stars 252 forks source link

Error while creating index for vector database table #1550

Closed aradhya-tiwari closed 2 months ago

aradhya-tiwari commented 2 months ago

I am trying to create a vector store using following command:-

CREATE TABLE IF NOT EXISTS
  embeddings (
    text TEXT,
    metadata TEXT,
    embedding F32_BLOB (1536)
  );

CREATE INDEX IF NOT EXISTS documents_idx ON documents (
  libsql_vector_idx (embedding, 'type=diskann', 'metric=cosine')
);

and getting this error

Error: SQLITE_UNKNOWN: SQLite error: unexpected vector column type: F32_BLOB (1536) at window.drizzle (https://turso.tech/_next/static/chunks/7061-c994b6446462fd0a.js:1:28549) at async Object.raw (https://turso.tech/_next/static/chunks/74f8cf84.b675ca02858c4173.js:400:7686) at async Object.raw (https://turso.tech/_next/static/chunks/74f8cf84.b675ca02858c4173.js:400:3481) at async executeSqlQuery (https://turso.tech/_next/static/chunks/74f8cf84.b675ca02858c4173.js:258:19680)

I have updated my default group version to vector using

turso group create default --version vector

I also found that when I execute turso db show the version is ector instead of vector

sivukhin commented 2 months ago

Hi @aradhya-tiwari!

I think the error is due to space between F32_BLOB type name and dimension arguments. Can you try DDL like this:

CREATE TABLE IF NOT EXISTS
  embeddings (
    text TEXT,
    metadata TEXT,
    embedding F32_BLOB(1536)
  );

About ector version - don't worry. You actually have vector DB version - we just cutting first symbol (will fix this just to avoid such confusion)

aradhya-tiwari commented 2 months ago

yeahh silly mistake! actually was executing commands in turso SQL runner and didn't see the space.