dosaboy / searchkit

Apache License 2.0
1 stars 2 forks source link

Optimise data transfer between principle and worker procs #17

Closed dosaboy closed 2 months ago

dosaboy commented 2 months ago

SearchResultMinimal is used to transder results between the worker processes and the main collector. As a result it must be a small as possible in order to keep transfer fast and memory footprint low. There were several unnecessary variables being stored and duplicated in this object that have now been removed maing the transfer faster and using less memory. Transfers are also batched so as to reduce interruption on searches.