Main methods: make runs reproducible, reduce main churn

[voj]

Oakley is using main methods a lot and is having these problems:

• They make it hard to reproduce runs from toshi and it's easy to miss a parameter. See PythonGatewayJsonRunner for a way forward.
• We want to be able to switch between different configurations to reproduce experiments
• Modifying the main method directly leads to churn in git
  • One thing to look at is to fix /tmp or TEST as the output/input folder