testing script for gpkit-models

[whoburg]

testing.py assembles all tests into a TestCase and runs it.

This is ready for review. There are a few remaining issues that I would like @bqpd's help addressing:

• [ ] There appear to be a number of issues related to me probably doing imports incorrectly. Symptoms:
  • my pep8 complains about relative imports like import beam
  • That said, I can't get from . import beam to work
  • When I run from testing import run or from testing import beam in ipython, the beam test appears to execute (it shouldn't run the test when I only import). I can't figure out why.
  • When I run testing.py, only 4 tests run, not 5 (beam is missing)
  • Semi-unrelated import question: should there be an __init__.py in aircraft?
• [ ] Test for all solvers?
  • currently, testing.py only runs the test() method for the default solver. Should we test for all solvers like in GPkit?
• [ ] verbosity control
  • Currently running testing.py results in a lot of verbose printout. We need a good way to eliminate this (and recommendations for how future test() methods should handle printing

[bqpd]

1) Yeah, you can only do local imports with the from x import y syntax 2) Probably yes, though Jenkins will do this for you anyway with the way it's set up. 3) capture stdout and log it, as the examples do?

[whoburg]

1) but from . import beam gives me a ValueError: Attempted relative import in non-package. I must be missing something - the behavior I described above (with beam printing/solving unexpectedly) is very strange.

2) good point

3) sounds reasonable -- is there gpkit code we can borrow to make this easy? And should we also advocate some verbosity best practices for test() methods?

[bqpd]

1) does beam exist in __init__.py? 3) Yeah, there's some classes in t_examples.py!

[whoburg]

1) yes, good point, __init__.py does currently run from beam import Beam, but removing that doesn't solve the problem.

The problem is that from testing import run or even import testing currently causes a Beam to get created, solved, and a bunch of output is printed.

The problem appears related to the fact that my beam.__file__ currently returns '... /gpkit/docs/source/examples/beam.pyc', even when working in the model library repo. Indeed, the version of Beam in docs/examples does a bunch of nasty printing.

Any idea why that's the beam being imported? I haven't been able to track it down.

Also, do we need to have two separate Beams? Shouldn't the docs/examples version somehow just use the one from models library?

[whoburg]

@bqpd, any thoughts on the issues described above? Is part of it perhaps because we don't have a setup script yet?

[bqpd]

I added testing-by-model, based on this, to #62; take a look there!