epompeii
commented
1 month ago We have one metric which measures a worst case scenario and which is much larger than all the rest. This means that it dominates the graph
Yes, this is definitely less than ideal. I'm open to suggestions for better ways to plot a Report. For context this was added to overcome the "cold start problem" when viewing results. I figured it was easier to trim down the results than try to hunt down all the correct dimensions: https://github.com/bencherdev/bencher/issues/133
When this solution was originally created though, the format of the Report was very much in flux. With the Report format now more or less stabilized, I think it is worth reevaluating this design decision. A related step that I think would be of interest here is creating a Report results table view: https://github.com/bencherdev/bencher/issues/421
In criterion, we use groups with Ids that contain the method name and a size parameter: e.g. https://github.com/ratatui-org/ratatui/blob/main/benches/barchart.rs#L11-L27. It would be nice to make these groups, methods, and the size parameters be something that can used to select which methods to show.
Agreed! I think this would be covered by custom tags, once those are added: https://github.com/bencherdev/bencher/issues/240
In the mean time, have you taken a look at Pinned Plots? I need to add better docs around them, but Pinned Plots allow you to save queries and create a dashboard like interface. You can create a Pinned Plot for each of these parameter permutations to easily keep track of them. They also appear in the Perf Plot under the "Pinned" tab, so you can quickly pull them up.
Additionally, when there's a large amount of metrics, all with the same branch and machine name, this repeated info is overly verbose. While this looks pretty, a simple vertical list of the metric names with checkboxes would be much more usable (Discovering that the title box was clickable was a non-obvious user interface thing).
Very true. When a Report is selected these can very safely be deduplicated. I've created a tracking issue: https://github.com/bencherdev/bencher/issues/475
I also like the table view in the example below as an alternate (less pretty, more useful) view. Another thing that is difficult with the current design is easily seeing all the dimensions that are
P.s. it would be nice to have a copy image button on the chart. Especially if there's some way to include a legend that makes sense. In a previous life at a very large company I worked at we had similar info displayed underneath the chart as labels. For some inspiration, take a look at how CloudWatch displays this sort of thing (random image found on image search):
Have you taken a look at the Perf Plot Image yet? If you create a Perf Plot and then hit the "Share" button, a modal should pop up. This generates an image version of the plot. For example:
... and I now have a tracking issue for the y-axis running off the side of the image: https://github.com/bencherdev/bencher/issues/476 🙃
Is there some other type of image that you would want though?
joshka
https://bencher.dev/perf/ratatui-org
We have one metric which measures a worst case scenario and which is much larger than all the rest. This means that it dominates the graph
Turning that off gives another group of metrics that are all in the same area:
Turning those off gives us a bit more of a reasonable set of metrics:
In criterion, we use groups with Ids that contain the method name and a size parameter: e.g. https://github.com/ratatui-org/ratatui/blob/main/benches/barchart.rs#L11-L27. It would be nice to make these groups, methods, and the size parameters be something that can used to select which methods to show.
Additionally, when there's a large amount of metrics, all with the same branch and machine name, this repeated info is overly verbose. While this looks pretty, a simple vertical list of the metric names with checkboxes would be much more usable (Discovering that the title box was clickable was a non-obvious user interface thing).
P.s. it would be nice to have a copy image button on the chart. Especially if there's some way to include a legend that makes sense. In a previous life at a very large company I worked at we had similar info displayed underneath the chart as labels. For some inspiration, take a look at how CloudWatch displays this sort of thing (random image found on image search):