Motivation: the speed.yjit.org page is a useful summary of the progress our team is making. It's useful both internally and externally. I look at it all the time to get a quick idea of the progress we're making. I think it's very valuable to optimize the information we display on that page to maximize usefulness, and focus our attention on the metrics we care about the most.
[X] Bars comparing YJIT 3.3 vs 3.4dev
[x] Remove the bar for the interpreter, replace it with a horizontal line at 1.0
This will make space for more bars or more benchmarks
[x] Create a normalized memory usage graph with a similar format to the speedup graph, also to be included on the front page
This will be very valuable because it's a big blind spot for us right now. We don't pay much attention
to how memory usage changes over time, and can get blindsided by regressions.
Possibly, we should measure this for the 3.4 interpreter as well? It would be good to be able to detect
regressions there, and know what is or is not due to YJIT.
This should also have geomean columns as well
Ideally, this should be measured using "max RSS", which yjit-bench is able to track (feel free to copy the code).
[x] Tweak colors, styling, current theme not aesthetically pleasing
(I realize that style is very subjective)
White background, colors that "pop" more for the bars?
The legend rectangles could be smaller and better aligned with the text (the text looks too close to the bottom side?)
264
[x] We currently show the speedup value when hovering, if there was a way to write this value inside the bars (or at the top at an angle) in a small font, it might be better?
The hover setup doesn't work on mobile, it's also not immediately obvious for new visitors.
[x] if we end up with 3 bars for the memory graph (3.4 interp, 3.3 YJIT, 3.4 YJIT), then maybe we also want to have the same 3 bars for the speedup graph, for consistency. This would also give us some visibility into interpreter performance improvements and regressions.
258
@rwstauner I ended up writing lots of notes. Hope this is helpful :)
Motivation: the speed.yjit.org page is a useful summary of the progress our team is making. It's useful both internally and externally. I look at it all the time to get a quick idea of the progress we're making. I think it's very valuable to optimize the information we display on that page to maximize usefulness, and focus our attention on the metrics we care about the most.
Lower priority:
264
Note:
258
@rwstauner I ended up writing lots of notes. Hope this is helpful :)