Runtime distributions (ECDFs) summary and function groups

[nikohansen]

When processing a single algorithm, the link "Runtime distributions (ECDFs) summary and function groups" does not link to what I would expect and like to see, namely for each function group (and for all functions together) a figure showing the (aggregated) ECDF graph for this group in different dimensions.

What I see under the link is the result on single functions, AFAICS the same graphs as under "Runtime distributions (ECDFs) per function" (where I indeed would expect them to see) but organised in different figures. This organisation seems mostly useful to assess the testbed group labels, but less useful to assess the algorithm. I am not sure we should even keep it by default?

The name for figures should preferably refer to what the shown graphs themselves represent rather than to the way how graphs are distributed (or grouped together) in different figures.

[brockho]

Mainly the same as #1177, but for one algorithm. Both issues should be addressed at the same time (or shortly after each other).

[brockho]

Following our discussion on September 12, 2017, we decided that

• [ ] we should have the same structure of output, independent of whether 1 or more algorithms are compared
• [ ] we can display various sets of targets in the ECDF graphs for one algorithm (with varying dimension). Specifically, we add five graphs with five (sorted) subsets of the sorted target values list. These graphs never cross (like in the scaling figures for one algorithm).
• [ ] we should check whether the text of the link reflects the actual content behind it (maybe already solved)

[brockho]

Add-on from today's discussion: we could also imagine having the current graphs (one for each function) and the aggregated ECDF over all functions in the group in the background. This will allow us to train ourselves to aggregate and potentially disaggregate data (or at least understand which kind of data could be underlying a certain ECDF graph).