jmitchell
commented
7 years ago Travis CI could probably be configured to push performance metrics commits to this repo after every successful build.
EDIT: Maybe it's not a great idea to store all the profiling data in this repo. One of my early experiments making an eflame stack.out file for a hard Sudoku puzzle results in an 80 MB file. Storing summary statistics wouldn't be so bad, though. Another concern about doing the profiling on Travis CI is the compute resources may be too inconsistent.
- [ ] add encrypted OAuth GitHub token to Travis config to grant commit rights
- after successful build have Travis...
- [ ] run the custom mix task (see #12)
- [ ] move output to
./metrics/#{commit_hash} - [ ] create a metadata file in that directory that encodes...
- [ ] commit's branch
- [ ] timestamp (produced by Travis when the task runs)
- [ ] mix environment used during profiling
- [ ] commit the changes and push to GitHub (include
[skip ci]in the commit message to avoid an infinite loop)
If that works out, make a mix task to produce graphs from data in ./metrics. Integrate the mix task into the automated after_success, so Travis can also publish keep summary statistics and graphs up to date.
Once #12 is finished I'll have a fairly easy way to analyze performance of the codebase at a particular commit, but as changes are made to any Backtrex backend, implemented behaviour, or input problems per behaviour performance will fluctuate.
The motivations are twofold: (1) avoid performance regressions and (2) motivate me to add more optimizations. It would help to have ongoing graphs of performance metrics over time, ideally per commit to master.