search

TuringLang / TuringGLM.jl

Bayesian Generalized Linear models using `@formula` syntax.
https://turinglang.org/TuringGLM.jl/dev
MIT License
70 stars 7 forks source link

Track performance of `turing_model` #47

Closed storopoli closed 2 years ago

storopoli commented 2 years ago

Thanks for the nice improvements, @devmotion. It’ll be great if we can track performance regression for TuringGLM (e.g. using a notebook similar to tutorials), but that can be done in a separate PR.

Originally posted by @yebai in https://github.com/TuringLang/TuringGLM.jl/issues/45#issuecomment-1055540632

rikhuijzer commented 2 years ago

Would pretty printing some outputs to a log be a solution or do we need something more fancy such as a static website or an automatic performance report comment? Static website is gonna be difficult since this repo already has Documenter in the gh-pages branch and automatic comment is gonna be annoying because bot comments are always annoying

rikhuijzer commented 2 years ago

Maybe we can add one issue about performance and automatically add comments to that issue with the benchmark results and a ref to the PR or to a commit on a branch. That way people can unsubscribe from the issue and not get notifications but still see results by clicking on the issue cross-ref. Also, the benchmark history is easily visible by scrolling through the issue comments.

EDIT: Nah. Nevermind. Logs make the most sense because they can easily be accessed from PRs and commits. LoopVectorization also just uses the logs as far as I can tell

devmotion commented 2 years ago

We use BenchmarkCI in KernelFunctions: https://github.com/JuliaGaussianProcesses/KernelFunctions.jl/pull/399