search

lanterndata / tutorials

0 stars 1 forks source link

Tutorial webpage does not match jupyter notebooks #2

Open aubreyyan opened 3 weeks ago

aubreyyan commented 3 weeks ago

Consistently, the website with the tutorials instructs users to create an index in postgresql using the access method lantern_hnsw. image This is directly contradicted by the jupyter notebooks in https://github.com/lanterndata/examples/tree/main image And it is unclear which, if any, is correct.

After following the tutorials, neither access method works for me.

# python3 setup_embeddings.py
INFO:root:Connecting to ourdb on localhost:5432
INFO:root:Table movies does not exist, creating it
INFO:root:Found 3883 embeddings with dimensionality 32
INFO:root:Creating L2 Norm index on movies
Traceback (most recent call last):
  File "/root/projects/lantern_movie_recommender/setup_embeddings.py", line 42, in <module>
    ourdb.execute(f"CREATE INDEX ON {constants.MOVIES_TABLE_NAME} USING lantern_hnsw (vector dist_l2sq_ops) WITH (dim={dimensions});")
psycopg2.errors.UndefinedObject: access method "lantern_hnsw" does not exist
# python3 setup_embeddings.py
INFO:root:Connecting to ourdb on localhost:5432
INFO:root:Table movies does not exist, creating it
INFO:root:Found 3883 embeddings with dimensionality 32
INFO:root:Creating L2 Norm index on movies
Traceback (most recent call last):
  File "/root/projects/lantern_movie_recommender/setup_embeddings.py", line 46, in <module>
    ourdb.execute(f"CREATE INDEX ON {constants.MOVIES_TABLE_NAME} USING hnsw (vector dist_l2sq_ops) WITH (dim={dimensions});")
psycopg2.errors.UndefinedObject: access method "hnsw" does not exist

to reproduce, feel free to reference my source code. https://github.com/aubreyyan/lantern_movie_recommender

dqii commented 2 weeks ago

Hi! Sorry for these issues and thanks for letting us know.

lantern_hnsw is the correct access method. I just made these changes.

I also simplified the docs to use Lantern Cloud URLs for simplicity, although the self-hosting instructions still work.

Your issue with lantern_hnsw seems to be that the Lantern extension isn't loaded in the database.

In your setup.sh, can you change

sudo -i -u postgres psql -U postgres -c 'CREATE EXTENSION lantern'

to

sudo -i -u postgres psql -U postgres -d ourdb -c 'CREATE EXTENSION lantern'

I believe that should fix your error.