search

deshaw / versioned-hdf5

Versioned HDF5 provides a versioned abstraction on top of h5py
https://deshaw.github.io/versioned-hdf5/
Other
75 stars 18 forks source link

Get airspeed velocity benchmarks working again (PyInf#12733) #343

Open ArvidJB opened 2 weeks ago

ArvidJB commented 2 weeks ago

Not sure when the benchmarks stopped working... We used to automatically populate https://deshaw.github.io/versioned-hdf5/benchmarks on every commit by running airspeed velocity.

Let's get this working again!

Also, we may want to update the benchmarks https://deshaw.github.io/versioned-hdf5/master/performance.html All of this is probably much faster now?

asmeurer commented 4 days ago

The benchmarks were never run automatically. I just ran them on my computer every once in a while. asv automatically includes all commits when you run it. It's hard to run long term benchmarks on CI because you can't really be sure you have a consistent runtime across runs.

asmeurer commented 4 days ago

You can use CI to run a comparison benchmark between the PR and master, because the runs will be consistent for the same runner. We do this in SymPy https://github.com/sympy/sympy/blob/0710d784347042c54270fd562a47317607b9be30/.github/workflows/runtests.yml#L482 https://github.com/sympy/sympy/blob/master/.github/workflows/comment-on-pr.yml