search

AlexAltea / milli-py

Python bindings for Milli, the embeddable Rust-based search engine powering Meilisearch
MIT License
123 stars 2 forks source link

UserError(MaxDatabaseSizeReached) #5

Closed zintyminty closed 1 year ago

zintyminty commented 1 year ago

I have tried adding 4k items to Milli but I'm getting this error:

pyo3_runtime.PanicException: calledResult::unwrap()on anErrvalue: UserError(MaxDatabaseSizeReached)

My code:

movies_list = [] movies = get_movies() for movie in movies: movies_list.append({"id": movie.id, "title": f"{movie.title} {movie.release_year}", "imdb": movie.imdb}) index.add_documents(movies_list)

Any idea what is going wrong?

mardix commented 1 year ago

Why was this closed with no answers or explanation?

AlexAltea commented 1 year ago

@zintyminty You should specify the maximum size of the database when creating the Index, see: https://github.com/AlexAltea/milli-py/blob/master/docs/index.md#constructor

From the code I see you are loading the IMDB database, from my own tests 2 GiB or 4 GiB works pretty well: https://github.com/AlexAltea/curator/blob/master/curator/databases/imdb.py#L86 Indexing the entirety of IMDB will take about 1.1 GB.

Please reopen the issue if this does not fix your issue.