Open AlexAltea opened 1 year ago
Couldn't the tuple or the other list types be converted to list ([{...}, ...]) before adding the documents to Index.add_documents?
Yes, you can always do that, but it might have some performance impact. Imagine you want to index a million documents which happen to be a tuple
, or dict_values
.
Doing list(...)
causes everything to be copied from the original tuple into a new list, and then processed by Milli.
What I'm trying to do is have Milli's DocumentBatchBuilder
receive the documents straight from the tuple/dict_values.
ok. make sense.
E.g.
tuple
,dict_values
.Otherwise there might be performance loss during to needless conversions.