Closed zintyminty closed 1 year ago
Why was this closed with no answers or explanation?
@zintyminty You should specify the maximum size of the database when creating the Index, see: https://github.com/AlexAltea/milli-py/blob/master/docs/index.md#constructor
From the code I see you are loading the IMDB database, from my own tests 2 GiB or 4 GiB works pretty well: https://github.com/AlexAltea/curator/blob/master/curator/databases/imdb.py#L86 Indexing the entirety of IMDB will take about 1.1 GB.
Please reopen the issue if this does not fix your issue.
I have tried adding 4k items to Milli but I'm getting this error:
pyo3_runtime.PanicException: called
Result::unwrap()on an
Errvalue: UserError(MaxDatabaseSizeReached)
My code:
movies_list = [] movies = get_movies() for movie in movies: movies_list.append({"id": movie.id, "title": f"{movie.title} {movie.release_year}", "imdb": movie.imdb}) index.add_documents(movies_list)
Any idea what is going wrong?