Use airspeed velocity to track performance changes

[ketch]

Airspeed velocity is a tool for tracking performance of the code. Basically, it checks out a series of commits and runs some benchmarks on each one, then shows you how the performance has changed over time.

I managed to get it working on my computer by the following steps:

setup:

mkdir asv_pyclaw
cd asv_pyclaw
asv quickstart

Then replace your asv.conf.json file with this: https://gist.github.com/ketch/00e4f4cd33904bbfe6e2

Finally, you can run the benchmarks and see the output with this:

cd ~/asv_pyclaw
asv run v5.0.0..master  
asv publish
asv preview

It would be great to have a public-facing server that automatically does this and hosts the results.

[mandli]

I keep getting

IOError: trying to symlink /Users/mandli/src/clawpack/asv_pyclaw/clawpack/riemann/src/python/riemann: which does not exist

when trying the instructions. Any idea why this might happen?

[ahmadia]

Your master Clawpack and Riemann repositories are out of sync. Try bumping to master on both.

On Sun, Nov 16, 2014 at 8:30 PM, Kyle Mandli notifications@github.com wrote:

I keep getting

IOError: trying to symlink /Users/mandli/src/clawpack/asv_pyclaw/clawpack/riemann/src/python/riemann: which does not exist

when trying the instructions. Any idea why this might happen?

— Reply to this email directly or view it on GitHub https://github.com/clawpack/pyclaw/issues/476#issuecomment-63228817.

[mandli]

Working now, thanks for knowing what the problem was so quickly @ahmadia.

Back to the issue, do we have a machine that could be dedicated to this and possibly have the output linked to the clawpack.org website?

[ketch]

I have machines, but they are behind the KAUST proxy so I can't serve pages publicly. Maybe they could be set to automatically upload results somewhere...

[mandli]

Maybe it's enough to compare subsequent release versions, then we can just include it in the static docs.

[ketch]

That would at least be a start.