milvus-io / milvus-haystack

Apache License 2.0
6 stars 12 forks source link

get_all_documents_generator does not implement what it promises to #8

Open apohllo opened 7 months ago

apohllo commented 7 months ago

The method description says:

Get documents from the document store. Under-the-hood, documents are fetched in batches from the
        document store and yielded as individual documents. This method can be used to iteratively process
        a large number of documents without having to load all documents in memory.

while the implementation is:

for doc in self.get_all_documents(
            index, filters, return_embedding, batch_size, headers
        ):
            yield doc

I think NotImplementedError should be raised for this method.