Add summary dashboard for PerfKitBenchmarker.

GoogleCloudPlatform / PerfKitExplorer

PerfKit Explorer is a dashboarding and performance analysis tool built with Google technologies and easily extensible. PerfKit Explorer is licensed under the Apache 2 license terms. Please make sure to read, understand and agree to the terms of the LICENSE and CONTRIBUTING files before proceeding.

Apache License 2.0

267 stars 64 forks source link

Add summary dashboard for PerfKitBenchmarker. #172

Closed yuyantingzero closed 9 years ago

cmccoy commented 9 years ago

Please remove the mongodb benchmark.
(at least) the netperf link does not work for me.
Consider adding some more netperf metrics.

jmuharsky commented 9 years ago

The SQL statements provided in the json dashboard are still using the {{dataset_name}} and {{table_name}} placeholders.

yuyantingzero commented 9 years ago

@cmccoy :

Removed mongodb benchmark.
The link points to dive-in dashboards, which are not ready in this PR. I will fix the links later in next PRs.
Added TCP_RR for netperf. I will leave the dashboard file for now, only updating the dashboards on the experiment Explorer site. And download and update the json file later in this PR.

@jmuharsky : The SQL statement in this PR uses {{dataset_name}}, {{table_name}}. In the README.md file, there are instructions to replace dataset_name, table_name with target dataset and table. I can use samples_mart, results directly if you think that is enough for most users. I will send you the deployed preview dashboards offline.

cmccoy commented 9 years ago

@yuyantingzero: Thanks. What is the criterion for including a result from a benchmark in this summary page?

yuyantingzero commented 9 years ago

@cmccoy I am trying to include 1 or 2 most important metric per benchmark. The dive-in dashboards contains all metics with parameters. Feel free to point out any different metric should be added into summary dashboard in your mind.

cmccoy commented 9 years ago

Could you note that this uses the default machine type in the header for the first container?
How are new clouds going to be handled? Could you use pivots?
There are two "netperf" rows - maybe just have one label for the netperf group?

Otherwise, LGTM.

jmuharsky commented 9 years ago

@yuyantingzero We shouldn't be varying from the samples_mart.results pattern, so I think fixing these is the right thing to do. We can instruct the user to change those values if they should upload to a different dataset/table, but our tooling should guide them towards these values.

jmuharsky commented 9 years ago

Recommend a more descriptive title than _PerfkitBenchmarker -- something like PerfKit Benchmarker Summary Dashboard (Sample).

yuyantingzero commented 9 years ago

@jmuharsky Addressed all comments. @cmccoy Used pivots in summary dashboard now, but since query has a different set of results, the color schema is different in each widget, any suggestions?

cmccoy commented 9 years ago

LGTM modulo missing data.

jmuharsky commented 9 years ago

Recommend moving the legends to top, bottom or inline/right, and providing enough "left" on the chart area to account for the x axis.

yuyantingzero commented 9 years ago

Addressed all comments, PTAL @jmuharsky

yuyantingzero commented 9 years ago

@jmuharsky Updated color and ordering.

jmuharsky commented 9 years ago

LGTM