search

moby / buildkit-bench

Set of benchmarks for BuildKit
https://moby.github.io/buildkit-bench/
Apache License 2.0
0 stars 2 forks source link

unexpected metrics for BenchmarkBuildSecret #124

Open crazy-max opened 6 hours ago

crazy-max commented 6 hours ago

Since https://moby.github.io/buildkit-bench/#/result/20240918-081354, BenchmarkBuildSecret: https://github.com/moby/buildkit-bench/blob/114bb20865cb2fb3f2ff30b8fc5faccd6ebab28d/test/build_test.go#L76-L93

does not return what we would expect.

Previously: https://moby.github.io/buildkit-bench/#/result/20240917-213205

image

Current: https://moby.github.io/buildkit-bench/#/result/20240918-081354

image

We should see the regression between v0.12.5 and v0.13.0 related to https://github.com/moby/buildkit/pull/4624 as shown in previous benchs:

image

But we don't since https://moby.github.io/buildkit-bench/#/result/20240918-081354

image

Changes we made between 20240917-213205 and 20240918-081354 doesn't seem related:

https://github.com/moby/buildkit-bench/compare/88ac7cbffcd401ca5344d3a14c840c8a66910da5...1b615403b7f988a1041d5a4a10287848ccd2d215