I stumbled across this Slack thread recently when I was trying to measure small kernels with nvbench and got fluctuating results.
As @senior-zero notes in the thread, the variance of the minimum runtime is usually smaller than that of the average runtime.
So it would be very useful if nvbench would provide a way to generate these values directly.
We could also report additional metrics, such as max or median runtime.
I stumbled across this Slack thread recently when I was trying to measure small kernels with nvbench and got fluctuating results. As @senior-zero notes in the thread, the variance of the minimum runtime is usually smaller than that of the average runtime.
So it would be very useful if nvbench would provide a way to generate these values directly. We could also report additional metrics, such as max or median runtime.