Open freef4ll opened 11 months ago
Docker scripts have been updated, but having issues with thresholds:
+ export PATH=/root/.tools:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ PATH=/root/.tools:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ cd Benchmarks
+ swift package --disable-sandbox --scratch-path .build/5.8/ --allow-writing-to-package-directory benchmark --format metricP90AbsoluteThresholds --path Thresholds/5.8/
error: 'benchmarks': unknown package 'swift-prometheus' in dependencies of target 'PrometheusBenchmarks'; valid packages are: 'code', 'package-benchmark'
Split the benchmark code into the benchmark definition and separate files for the benchmarks under test
I have done this, and there are 2 implementations for Counter to see on which is the preferred one (one captures the allocation but is slower due to return of the closure).
I have fixed the SPM error and thresholds are being generated fine now; just need to decide of which code style implementations to keep.
@FranzBusch , could you have a look at the PR code style that is preferred? Counter #2
doesn't capture the allocation of the counter it self and is slower.
@swift-server-bot add to allowlist
@swift-server-bot test this please
I'm not sure I understand on what happened with https://ci.swiftserver.group/job/swift-prometheus-soundness-prb/31/ and what other PR state https://ci.swiftserver.group/job/swift-prometheus-soundness-prb/buildTimeTrend ; especially runs 27 and 25 which report the same result.
EDIT: fixed
Anything we can do to help move this forward?
From my perspective this looks fine, any concerns @FranzBusch @fabianfett ?
@swift-server-bot test this please
Failures because of missing baseline: Could not find any matching absolute thresholds at path [Thresholds/swift-5.7.3-RELEASE/], failing threshold check.
@fabianfett I think had some opinions before we merged here?
Motivation
Adding benchmark tests in order to aid development and avoid performance regressions.
Benchmark is added in the same fashion as for swift-nio and swift-certifcates.
Modifications
Benchmarks are added for incrementing a counter, gauge, histogram and an export of 5000 metrics from the esystem.
How to execute benchmark
In Xcode can open the project like and use Instruments to profile:
Result
Benchmarks are available, and performance difference between v1 is great which only managed to increment 2-3k times per second.