search

ropensci / unconf17

Website for 2017 rOpenSci Unconf
http://unconf17.ropensci.org
64 stars 12 forks source link

Testing and reporting performance regressions #90

Open jeroen opened 7 years ago

jeroen commented 7 years ago

I sometimes feel that some of my code seems to have gotten faster or slower, even though I don't know why. It would be nice to have better performance diagnostics to benchmark functionality in between commits or releases.

This cannot fully be automated because functionality can actually change in between releases and might not be comparable. Moreover performance is random due to hardware and other fluctuations.

Yet some sort of scorecard (similar to covr) to give some sense of how commits affect performance might be useful. Perhaps something on top of testthat reporting timings and highlight the biggest regressions or optimisations.

gaborcsardi commented 7 years ago

I think it would be nice to have new, modern web app to follow code speed, something like https://github.com/tobami/codespeed but better, with a proper HTTP API.

jimhester commented 7 years ago

https://github.com/ropensci/unconf17/issues/56 is the same thing, codecov.io is going to be getting a plugin system in the not too distant future, so I am planning on using it to do this, testthat already has a reporter that will time tests for you so most of the pieces are already there.