Would this package be open to alternative calculations of the "median" run? Currently, it chooses the run with the median TTFCPUIDLE value, but that may not necessarily be the median SI or TTI value either. Depending on which value the consumer is most concerned with testing, I could see misleading results by always basing the median run off TTFCPUIDLE.
I'm wondering if it would be helpful to provide config flags to specify which nmetric to calculate the median run using.
I'm also tossing around whether I think it would be useful to report the average for each metric independently or not.
Would this package be open to alternative calculations of the "median" run? Currently, it chooses the run with the median
TTFCPUIDLE
value, but that may not necessarily be the medianSI
orTTI
value either. Depending on which value the consumer is most concerned with testing, I could see misleading results by always basing the median run offTTFCPUIDLE
.I'm wondering if it would be helpful to provide config flags to specify which nmetric to calculate the median run using.
I'm also tossing around whether I think it would be useful to report the average for each metric independently or not.
Thanks!