The indexer would return a list of entries, and these entries were sent across process boundaries regularly (in the store, and in the reindex action), which would cause their memory usage to swell.
Returning a stream alleviates this pressure, and reduces the memory used by on the lexical project from 425Mb to 290Mb, a fairly significant savings.
The indexer would return a list of entries, and these entries were sent across process boundaries regularly (in the store, and in the reindex action), which would cause their memory usage to swell.
Returning a stream alleviates this pressure, and reduces the memory used by on the lexical project from 425Mb to 290Mb, a fairly significant savings.
Fixes #597