search

zarr-developers / numcodecs

A Python package providing buffer compression and transformation codecs for use in data storage and communication applications.
http://numcodecs.readthedocs.io
MIT License
124 stars 85 forks source link

Continuous benchmarking with ASV #126

Open alimanfoo opened 5 years ago

alimanfoo commented 5 years ago

It would be great to implement ASV or similar so we catch any changes that introduce a performance regression.

jakirkham commented 1 year ago

@joshmoore this was something we were looking at right?

joshmoore commented 1 year ago

@jakirkham, yes, during the last GSOC round. There are a few decisions to be made and then various functions to be written, but pretty straight-forward.

jeromekelleher commented 1 year ago

Just FYI, it's worth having a look at upstream discussions about the future of the project (e.g. https://github.com/airspeed-velocity/asv/issues/1219 ) before committing to ASV

joshmoore commented 1 year ago

Oh wow. Thanks for the heads up, @jeromekelleher! (Yet another reason it was sad to have missed the summit.) I'll keep an eye on it, but in our testing with asv so far (for zarr-python) there was already one "speed regression" identified. (By another "j" programmer who shall rename nameless.) So it might make sense to get something in place soonish, with the understanding that we will be up for jumping ship when the time comes.

JackKelly commented 11 months ago

Just following up on this: Were any benchmarks implemented for numcodecs? If so, did the benchmarks use ASV?

it's worth having a look at upstream discussions about the future of the project (e.g. airspeed-velocity/asv#1219 ) before committing to ASV

For context: I'm currently researching benchmarking tools for benchmarking zarr-python. ASV looked good. Until I saw that discussion!

joshmoore commented 11 months ago

In the context of GSOC asv was only implemented for zarr-python and there only superficially.