As we are working on CI with runners using GPUs, @marshallmcdonnell raised the value of regression testing for performance. @ckendrick further raised the issue that inception's GPUs are kept busy so performance measurement must be robust in the face of resource contention.
It would be great to get an accurate idea of performance trends as the code itself changes, also to get early warning of slow code being introduced. Perhaps some per-test history is possible so that one may pinpoint the affected behaviors.
As we are working on CI with runners using GPUs, @marshallmcdonnell raised the value of regression testing for performance. @ckendrick further raised the issue that inception's GPUs are kept busy so performance measurement must be robust in the face of resource contention.
It would be great to get an accurate idea of performance trends as the code itself changes, also to get early warning of slow code being introduced. Perhaps some per-test history is possible so that one may pinpoint the affected behaviors.