Open jsearles57 opened 5 years ago
Yeah, I would also like to see this one working, since I cannot manage to create a custom schema for google benchmark. The referenced one seems not to work...
I was able to use the reference benchmark schema from the stackoverflow post:
benchmark(
altInputSchema: '''{
"description": "Google Benchmark JSON schema",
"failure": { "value": true },
"type": "object",
"properties": {
"benchmarks": {
"type": "array",
"items": {
"type": "object",
"properties": {
"name": { "type": "name" },
"real_time": { "type": "result" },
"cpu_time": { "type": "result" },
"iterations": { "type": "parameter" }
}
}
}
}
}''',
inputLocation: 'benchmark_results.json',
schemaSelection: 'customSchema',
thresholds: [[$class: 'PercentageThreshold', percentage: 1.0, testGroup: '', testName: 'cpu_time']],
truncateStrings: false
)
this is the pipeline call, where I earlier executed google benchmark with the arguments to output results into a benchmark_results.json file.
The problem is that thresholds are not really usable, positive improvements are not allowed (e.g. faster execution in latest build compared to previous).
Add google benchmark schema. An example of what the schema might look like is at https://stackoverflow.com/a/53226336