Bump vendor/benchmark (the newer version has better comparison and reporting scripts)
Add a new benchmark for rendering a map to help tracking TTRC (time to render completion)
Increase the iteration and repetition count for benchmarks to get more stable results
Disable benchmark for MacOS debug CI builds because it was causing timeouts due to increased iteration/repetitions (it uses a very slow SW renderer)
Add benchmark baselines for Android and iOS
Add a helper script to compare benchmark results to baseline, mark the benchmark as failed if the average time of any test is more than 10% above the baseline. The threshold is 10% to avoid false failures. Most of the time the results are within few percent from the baseline but in some cases the difference is larger. The reason could be that some devices run slower than others.
Unrelated change:
Store rebaselined metrics as build artifacts for Android render test build
Changes in this PR:
Unrelated change:
Example results for Android and iOS.
Fixes: mapbox/mapbox-gl-native-team/issues/39