Non-Deterministic Test Failures

[danielenricocahall]

The parametrized tests in integration/test_end_to_end will have non-deterministic test failures, usually due to a port being occupied already. This makes things flaky and hard to diagnose when there are real problems.

Potential solutions:

• Investigate socket/Flask servers not closing down correctly?
  • Are the connections staying alive incorrectly?
• Use a global random object (i.e; a pytest fixture with session scope?) which samples without replacement for producing each port while testing, rather than randomly selecting one for each test.

[danielenricocahall]

This is now resolved. Closing