Current implementation creates a new goroutine every 250ms (default local aggregation interval) and the goroutine gets destroyed after the aggregated values are written to db. This might be a cpu intensive operation. A brute force solution could be to run the accumulatorJob as part of the Run thread but then aggregation might slow down if processing takes more than the aggregation interval. A suggested solution is to have a few (2-3) active goroutines pool and use whichever is available to process
Current implementation creates a new goroutine every 250ms (default local aggregation interval) and the goroutine gets destroyed after the aggregated values are written to db. This might be a cpu intensive operation. A brute force solution could be to run the
accumulatorJob
as part of theRun
thread but then aggregation might slow down if processing takes more than the aggregation interval. A suggested solution is to have a few (2-3) active goroutines pool and use whichever is available to processhttps://github.com/Bisonai/orakl/blob/a95bf34cfc83a776338cf12aba10095322066612/node/pkg/fetcher/accumulator.go#L33