Open siddharthteotia opened 2 years ago
good idea. we also had some feedback from the community on this. but the stats definition is the first to sort out.
I agree. Measurement is probably ok but correct propagation, aggregation and reflecting at different stages is also something that needs to be thought about
In the current engine, we measure threadCpuTime metric at the BaseCombineOperator level. Aggregated metric is sent to servers and broker aggregates across servers and includes in query response stats. We need to do something similar (granularity etc to be discussed) in the multi-stage engine.