In this PR, we combine data in the comparison view for multiple experiment ids and trial ids.
We essentially stop using expId and trialId to distinguish data sets from now on.
Instead, we rely solely on commit ids.
The user interface of ReBenchDB is all around commit ids, so, it makes sense to combine multiple experiments that provide data for a specific revision, and the assumption is that there's no change in the setup, environment, benchmarks, or executors between different experiments on the same source revision.
As a consequence, the plots to visualize warmup data may need to show data from multiple trials/experiments.
The profiling info now can be a list of profiles, which are all shown under the corresponding revision.
We do not track invocation numbers anymore in the data for computing comparisons.
The reason is that we assume invocations to be independent, and it doesn't matter in which order they appeared.
This makes it simple to combine invocations from different experiments and trials, which are for the same experiments.
In this PR, we combine data in the comparison view for multiple experiment ids and trial ids. We essentially stop using expId and trialId to distinguish data sets from now on. Instead, we rely solely on commit ids. The user interface of ReBenchDB is all around commit ids, so, it makes sense to combine multiple experiments that provide data for a specific revision, and the assumption is that there's no change in the setup, environment, benchmarks, or executors between different experiments on the same source revision.
As a consequence, the plots to visualize warmup data may need to show data from multiple trials/experiments. The profiling info now can be a list of profiles, which are all shown under the corresponding revision.
We do not track invocation numbers anymore in the data for computing comparisons. The reason is that we assume invocations to be independent, and it doesn't matter in which order they appeared. This makes it simple to combine invocations from different experiments and trials, which are for the same experiments.
This fixes #158 and #159.