lanterndata / lantern

PostgreSQL vector database extension for building AI applications
https://lantern.dev
GNU Affero General Public License v3.0
790 stars 57 forks source link

ERROR: column does not have dimensions, please specify one #197

Closed min-mwei closed 1 year ago

min-mwei commented 1 year ago

Looks like there must be at least one row in the table before one could create a lantern index. Take the "hello world" example,

CREATE TABLE small_world (id integer, vector real[3]);
CREATE INDEX ON small_world USING lantern_hnsw (vector);

This above will fail with the message in the title. Of course if one add insert the create table, it works.

Thanks.

Ngalstyan4 commented 1 year ago

I see.

You are trying to create an index on an empty table. The problem is that the index does not know the dimension of the vectors you will be inserting.

The CREATE INDEX statement on an empty table should work fine once you explicitly state dimensions of the vectors in the index as below:

CREATE INDEX ON small_world USING lantern_hnsw (vector) WITH (dim = 3);

I think you expect to not need the dimension specification because you already specify the dimension of the real[] array in CREATE TABLE statement (real[3]) But postgres actually does not enforce dimensions on arrays and the3inreal[3]` ends up just being decorative/documentation. So, our index has no way of picking it up.

Does this fix the issue?

min-mwei commented 1 year ago

Thanks for the quick response, yes, this addresses this issue.