scikit-hep / pyhf

pure-Python HistFactory implementation with tensors and autodiff
https://pyhf.readthedocs.io/
Apache License 2.0
283 stars 83 forks source link

Add WS_Builder benchmarks #111

Open matthewfeickert opened 6 years ago

matthewfeickert commented 6 years ago

As pointed out in Issue #77, @vincecr0ft has made WS_Builder benchmarks at WorkSpaceBuilder. These need to be integrated into the pyhf benchmarking suite.

First thoughts on steps:

lukasheinrich commented 6 years ago

what we need for this is e.g. a setup where we can generate a) JSON specs and b) XML + ROOT files from a single source of truth

           source of truth (could just be the JSON spec, either from disk or generating a bunch on-the fly)
             /           \
     XML + ROOT   JSON spec (if needed)
            |            |
        ROOT HF         pyhf
            |            |
      results.json      results.json
             \           /
       validation by comparison
              benchmarking
matthewfeickert commented 6 years ago

Given this I should probably complete Issue #3 first.

matthewfeickert commented 6 years ago

For future reference, a solution to the benchmark naming problems I was running into with pytest-benchmark is to simply pass the --becnhmark-histogram option last.