anniesullie
commented
7 years ago @simonhatch and @eakuefner have been looking into the dashboard backend lately. Some ideas on how this could work:
- Actually make the cloud datastore backend available in some other context. I haven't looked into feasibility here, especially around whether we could make it read-only. I think the lack of indexes would make this painful, even if we could find a way to give them a read-only copy.
- Depending on how narrow the use-case is, make a custom report generation page. I think this is different from the sheriffing page @jessimb is working on because it's focused around understanding the effects of a particular cl, not around understanding the change in metrics over a longer time range. We could restrict this to a particular sheriffing rotation to make it easier to write a basic UI. If we gave the v8 team a page that implemented a few of these types of queries, is it something they'd be interested in hacking on directly?
- Have some kind of export of data to a dremel-able format. We'd want to restrict this to certain timeseries, hopefully based on a sheriffing rotation. We'd need to figure out the timing for this. Should it just be on-demand? If it's at regular intervals, how often would it need to happen to make the data usable?
simonhatch
natorion
eakuefner
Chromeperf has great data stored. Currently, when I want to do some analysis I need to individually open all the necessary benchmarks and bots. Crosschecking needs to be done manually. Simple use cases taken from actual day-to-day work:
1.) I wonder how much we improved on memory consumption after the fix has landed? Would be good to know for a blog post like https://v8project.blogspot.de/2016/10/fall-cleaning-optimizing-v8-memory.html.
With as simplistic query like (names made up btw)
SELECT min(avg_v8_heap), max(avg_v8_heap) FROM all_data WHERE test_suite="v8" AND benchmark="mobile_browsing" AND sub_test="NY times" AND revision="123456";
I could answer that question easily.
2.) We are currently tweaking the bucketting for the Runtime Callstats. The assumption is that a certain percentage of the work in one bucket, should show up in another with the "equal" (including error) amount.
Example (let's simply use a median over all the bots for good practice)
SELECT median(IC_Total), median(GC_Total) from all_data WHERE test_suite="v8" AND benchmark="top25" AND revision="123456"; //before change SELECT median(IC_Total), median(GC_Total) from all_data WHERE test_suite="v8" AND benchmark="top25" AND revision="123457"; //after change
Obviously this would also mean, that one can use myriad of charting/analysis tools to have a deeper, and more interactive look.