search

AlexAltea / milli-py

Python bindings for Milli, the embeddable Rust-based search engine powering Meilisearch
MIT License
123 stars 2 forks source link

Make Index.add_documents compatible with other list-like types #1

Open AlexAltea opened 1 year ago

AlexAltea commented 1 year ago

E.g. tuple, dict_values.

Otherwise there might be performance loss during to needless conversions.

mardix commented 1 year ago

Couldn't the tuple or the other list types be converted to list ([{...}, ...]) before adding the documents to Index.add_documents?

AlexAltea commented 1 year ago

Yes, you can always do that, but it might have some performance impact. Imagine you want to index a million documents which happen to be a tuple, or dict_values.

Doing list(...) causes everything to be copied from the original tuple into a new list, and then processed by Milli.

What I'm trying to do is have Milli's DocumentBatchBuilder receive the documents straight from the tuple/dict_values.

mardix commented 1 year ago

ok. make sense.