Initialize the Searcher, passing the index path pointing to the the 12 GB index.
searcher = Searcher(checkpoint=self._checkpoint, index=self._index_path, collection=Collection(path="empty.tsv"), config=config)
Initialize the IndexUpdater
index_updater = IndexUpdater(config=config, searcher=searcher, checkpoint=self._checkpoint)
Call IndexUpdater.update_searcher (or IndexUpdater.add) passing a new passage
Call Searcher.search querying for the newly added passage.
Result:
The newly added passage does not appear in the result.
Expected result:
The newly added passage must appear in the search result with the highest score.
Notes:
There is no call to persist_to_disk in the test.
The issue does not happen when testing with a small-sized index (~112KB).
Test environment:
Steps to reproduce:
searcher = Searcher(checkpoint=self._checkpoint, index=self._index_path, collection=Collection(path="empty.tsv"), config=config)
index_updater = IndexUpdater(config=config, searcher=searcher, checkpoint=self._checkpoint)
Result: The newly added passage does not appear in the result. Expected result: The newly added passage must appear in the search result with the highest score.
Notes: