This repository hosts code that supports the testing infrastructure for the main PyTorch repo. For example, this repo hosts the logic to track disabled tests and slow tests, as well as our continuation integration jobs HUD/dashboard.
This is very difficult to understand, at a glance, what the impact of a change is over models.
A simple improvement would be to report instead the relative speedup/slowdown for each model, and allow sorting by this quantity (see also https://github.com/pytorch/test-infra/issues/4136). You could then sort to see models that are most positively affected / least positively affected.
A common thing to do in this situation is to make a bar chart visually showing these difference. This would make it immediately visually clear what the dispersion of changes looks like.
ezyang
commented
1 year ago
Right now, the per model view looks like this
This is very difficult to understand, at a glance, what the impact of a change is over models.
A simple improvement would be to report instead the relative speedup/slowdown for each model, and allow sorting by this quantity (see also https://github.com/pytorch/test-infra/issues/4136). You could then sort to see models that are most positively affected / least positively affected.
A common thing to do in this situation is to make a bar chart visually showing these difference. This would make it immediately visually clear what the dispersion of changes looks like.