Create a web dashboard for tracking test results

[gsarma]

With @travs:
We were discussing having some kind of dashboard for summarizing recent tests across all repos. There might be pre-existing solutions for this, but basically we want a snapshot of how all of the builds are doing. This also relates to having timing information on the tests suites themselves- would be nice to quickly catch if some tests are taking a particularly long time (#174).

@rgerkin @slarson

[gsarma]

CC:

@cheelee

[pgleeson]

FYI, there's a dashboard of sorts on OSB here: http://www.opensourcebrain.org/status (slow to load) which checks results of tests across GitHub repos, incl some OpenWorm ones:

Only the rightmost column is automatically generated (star ratings for support are manually changed), but the plan is to integrate it further with OMV, where tests for running models in multiple simulators each have their own jobs on TravisCI, e.g. https://travis-ci.org/OpenSourceBrain/GranuleCell

[MichaelCurrie]

Travis CI itself provides such a dashboard, including timing:

At https://travis-ci.org/openworm/:

[cheelee]

The TravisCI interface should be a good start. @MichaelCurrie Is there a way to specify a range for expected timing, and have the system mark it out (say in Orange) as an anomalous time taken?

This is orthogonal to a performance regression dashboard, but could be a first-cut mechanism by we could decide if we want to look into the performance regression reports in more detail.

[MichaelCurrie]

@cheelee I don't think so; not that I know of nor from a cursory bit of searching.

FYI doing some googling, there appears to be a third-party dashboard tool, CI Status that might be worth exploring here if you really want extra customization.

[cheelee]

@MichaelCurrie Awesome, thanks! I'll check it out!

[rgerkin]

+1 on the OSB-style dashboard, especially when your tests contains more output than pass/fail.

[slarson]

We've had a few versions of this, and there are plans for some new ones. For now, closing.