ordo-one / package-benchmark

Swift benchmark runner with many performance metrics and great CI support
Apache License 2.0
326 stars 25 forks source link

Allow passing of state between startup/shutdown code and the actual benchmark inside the closure #252

Open hassila opened 6 months ago

hassila commented 6 months ago

Would be nice to streamline this a bit;

API suggestion:

it would be extremely cool, if we could return something from the setup hook that is then passed to the measure closure and the teardown hook

Like this:

    public convenience init?<each Setup>(
        _ name: String,
        configuration: Benchmark.Configuration = Benchmark.defaultConfiguration,
        closure: @escaping (Benchmark, repeat each Setup) async -> (),
        setup: () async throws -> (repeat each Setup),
        teardown: (repeat each Setup) async throws -> ()
    )

this way we don’t need to capture values outside the closures - to have them handy in setup and teardown :slightly_smiling_face:

an alternative spelling could be this:

    public convenience init?<each Setup>(
        _ name: String,
        configuration: Benchmark.Configuration = Benchmark.defaultConfiguration,
        closure: @escaping (((Benchmark) async throws -> ()) async throws -> ()) async throws -> ()
    )

which would result in:

Benchmark("my-cool-benchmark") { measure in
    // my setup code
    measure { benchmark in // handing control to the benchmark framework. closure can be invoked multiple times

    }
    // my teardown code
}

Could do stuff like e.g.

Benchmark("my-cool-benchmark") { measure in
    let pool = SomeConnectionPool()

    await withTaskGroup { taskGroup in
        taskGroup.addTask { await pool.run() }
        measure { benchmark in // handing control to the benchmark framework. closure can be invoked multiple times
            // do stuff with the pool
        }

        taskGroup.cancelAll()
    }
}