loonatick-src
commented
11 months ago Copying the benchmarks here:
import Benchmark
import BMDemos
import Foundation
let benchmarks = {
let coefficients = Array(stride(from: 0.1, through: 10.0, by: 0.1 * Double.pi))
Benchmark.defaultConfiguration = .init(
metrics: [.cpuTotal, .wallClock],
timeUnits: .automatic,
warmupIterations: 0,
scalingFactor: .mega,
maxDuration: .seconds(10)
)
Benchmark("Noop") { benchmark in
for _ in benchmark.scaledIterations {
}
}
Benchmark("blackHole(x)") { benchmark in
var x = 1.13413241e42
for _ in benchmark.scaledIterations {
blackHole(x)
}
}
Benchmark("sleep(1)",
configuration: Benchmark.Configuration(scalingFactor: .one)
) { benchmark in
sleep(1)
}
Benchmark("Naive polynomial") { benchmark in
let x = 1.13241241e34
for _ in benchmark.scaledIterations {
blackHole(polynomialNaive(x, coefficients: coefficients))
}
}
}And this is what the naive polynomial calculation looks like
public func polynomialNaive(_ x: Double, coefficients cs: [Double]) -> Double {
var result = 0.0
for (i,c) in cs.enumerated() {
result += c * pow(x,Double(i))
}
return result
}
hassila
OS: macOS 14.1 23B2073 arm64 Package benchmark version: 1.16.0 Swift version: Apple Swift version 5.11-dev (LLVM e131e99f323910c, Swift 1c3bd3e722f8030) Target: arm64-apple-macosx14.0
I had been using Benchmark 1.4 in an internal project, but on upgrading to 1.16.0 the printed results of the benchmarks all became absurdly small even though the benchmarks were taking the same amount of time to execute. I have created a reproducer for this here. It has four simple benchmarks
blackHolefunction called on aDoublevalue in a loop)sleep(1), sleep for one second, wall clock should be around 1 billion nanosecondsIf you clone the above repo and manually execute the benchmark target
$ swift run -c release DemoBenchmarksI get e.g. the following output for one of the benchmarks
In particular, one of the benchmarks is just
sleep(1), whose wall clock time is being correctly reported as 1 second $10^9$ nanoseconds. The naive polyomial benchmark is taking some 160 million ($1.6 \cdot 10^8$) nanoseconds. Not shown above are numbers for noop/empty loop (42 ns) andblackHole(800,000 ns). But when running using the package extension$ swift package benchmark --target DemoBenchmarksI get the following results
I see that
sleep(1)is correctly reported as 1000 msI did my best to verify that things are not actually being optimized out by inspecting the disassembled binary as well.