Closed pratikunterwegs closed 5 months ago
Yes, I'd recommend it. A problem with EpiNow2 is that the benchmarks take quite long to run (1-5 minutes) and so doing repeated testing with multiple tests inflates run times pretty quickly. Because of this we have small sample sizes for the repeated tests (n=5) and get quite lot of uncertainty in the benchmarks. I think it would work really well for less computation intensive code such as in styler.
Perhaps worth noting that the package is looking for a maintainer https://github.com/lorenzwalthert/touchstone/issues/31#issuecomment-1949634987 although code contributions from others are still supported.
Thanks @sbfnk - good to know - I'll look into adding this to the open PR.
Just looking into this and something I should have realised earlier is that it won't be useful to add this workflow in the current PR as the function names are changing as well. I could add the old names as aliases to the functions, but that would have to be reversed eventually. Possibly best to add this workflow after the current PR is merged.
This PR requests the addition of a GHA workflow for continuous benchmarking. This is expected to help avoid changes that reduce model run speed, which is important as the expected use case is 100s-1000s of runs.
An example can be found in {EpiNow2}. @sbfnk and @jamesmbaazam any feedback on your experience with using {touchstone} would be great - would you recommend using it?