search

scikit-hep / pyhf

pure-Python HistFactory implementation with tensors and autodiff
https://pyhf.readthedocs.io/
Apache License 2.0
281 stars 83 forks source link

Research if we can track performance metric like we track test coverage #128

Open lukasheinrich opened 6 years ago

lukasheinrich commented 6 years ago

Description

the initial pyhf code was purposefully kept very 'close to the metal' with a minimal amount of object orientation and lots of vectorized tensorlibrary code in order to keep performance high, but that design might also be constraining as we want to move towards more feature complete / extended histfactory functionality, where we might want to introduce more classes, more etc.

It would be good to be able to track performance as we do these refactorings so that we see whether a refactoring not only is functionally correct but also whether or not it affects performance.

We already have benchmarking code courtesy of @matthewfeickert

Is anyone aware of package that can track this on a commit-by-commit basis like coveralls.io does for test coverage.

Ideas @matthewfeickert @kratsg ?

matthewfeickert commented 6 years ago

I'm not really. A quick search pulled up codespeed but I'm not sure if that is actually the type of tools we need. I'll look more tonight.

kratsg commented 6 years ago

Idea: use cProfile (as this is python-based) and combine with snakeviz. Push cProfile output to github pages for continuous integration and have the html page get generated with snakeviz for hosting purposes.