ACEsuit / Polynomials4ML.jl

Polynomials for ML: fast evaluation, batching, differentiation
MIT License
12 stars 5 forks source link

Add Benchmarks via PkgBenchmarks #75

Closed tjjarvinen closed 10 months ago

tjjarvinen commented 10 months ago

This adds Benchmarks with PkgBenchmark.jl.

To use benchmarks you can

using BenchmarkTools
using PkgBenchmark
using Polynomials4ML

bench = benchmarkpkg(Polynomials4ML)
results = bench.benchmarkgroup

# You can search with macro "@tagged"
results[@tagged "derivative"]

To control benchmarks e.g. number of threads

t2 = BenchmarkConfig(env = Dict("JULIA_NUM_THREADS" => 2))
bench_t2 = benchmarkpkg(Polynomials4ML, t2)

To save benchmarks to a file

export_markdown("results.md", bench)

Compare to a different branch - note you need to commit your changes (also both branches need to have same benchmarks)

# current branch to "origin/main"
j = judge(Polynomials4ML, "origin/main")

Automatic benchmarks on pull requests

I added support for BenchmarkCI.jl, so by adding a label "Run Benchmarks" on PR CI will run benchmarks and compare to main branch. (Note this only works after this PR has been merged)

There is also an option to store benchmarks in the repo, but I have commented that out.

You should also be able to add more benchmarks yourself based on the benchmarks.jl file.

cortner commented 10 months ago

@CheukHinHoJerry -- when you have time, can you please take a look and familiarize yourself with this? If you are happy with this then I'll merge it and then we should make a new PR to add actual benchmarks.

CheukHinHoJerry commented 10 months ago

Happy for this to be merged as is. Thanks.

cortner commented 10 months ago

@tjjarvinen -- could you maybe add the instructions to a developer documentation page in the docs? Then I'll go ahead and make the merge.

cortner commented 10 months ago

Is this ready now?

tjjarvinen commented 10 months ago

yes

cortner commented 10 months ago

Thank you.