search

awadell1 / PkgJogger.jl

Take your packages for a jog!
https://awadell1.github.io/PkgJogger.jl/stable/
GNU General Public License v3.0
21 stars 0 forks source link

Add support for existing benchmarking frameworks #5

Open awadell1 opened 3 years ago

awadell1 commented 3 years ago

PkgBenchmarks.jl uses a single benchmark/benchmarks.jl file defining SUITE::BenchmarkGroup

This is similar to the example in https://github.com/JuliaCI/BenchmarkTools.jl/blob/master/benchmark/benchmarks.jl. But BenchmarkTools uses suite not SUITE

StaticArrays.jl uses PkgBenchmark's format but includes all benchmark/bench_*.jl files automatically (Yes, PkgJogger was inspired by this)

BaseBenchmarks.jl does not follow PkgBenchmark's format but does use SUITE

gdalle commented 6 months ago

Hi! Is there a reason why you chose not to rely on benchmark/benchmarks.jl? Even StaticArrays.jl does it that way, right? They just use that file to include every other benchmark/bench_*.jl

https://github.com/JuliaArrays/StaticArrays.jl/blob/07c12450d1b3481dda4b503564ae4a5cb4e27ce4/benchmark/benchmarks.jl#L1-L12

awadell1 commented 6 months ago

Mainly, I'm lazy and wanted PkgJogger to handle the bundling up bench_*.jl files into a single suite for me. It also let's PkgJogger provide the following features:

  1. Support for a nested benchmarks/ folders. i.e. bechmark/bench_bar/bench_foo.jl will be added to the suite
  2. Revise support does a bit more than just including the bench_*.jl file, by bundling bench_*.jl files internally, revise can track changes at the file level, instead of the suite level.
  3. Each bench_*.jl is wrapped in a module ensuring each file gets it's own global namespace.

Plus PkgJogger still plays nice with the other packages, just put this in your benchmark/benchmarks.jl instead:

using PkgJogger
using AwesomePkg
@jog AwesomePkg
SUITE = JogAwesomePkg.suite()

Does that help?