search

cloudflare / tls-tris

crypto/tls, now with 100% more 1.3. THE API IS NOT STABLE AND DOCUMENTATION IS NOT GUARANTEED.
Other
291 stars 51 forks source link

interop_test_runner: use Python 3 #170

Closed Lekensteyn closed 5 years ago

Lekensteyn commented 5 years ago

'python2' does not exist on macOS, it is called 'python2.7'. Porting to Python 3 is however more future-proof so do that instead. Rename the file such that it can be run with pytest.

Do not listen on a fixed host port and use a random port instead. Otherwise tests could fail if something is listening on those ports.

Note, with this modification, I noticed some ResourceWarnings after calling exec_run due to a Docker socket being leaked. These won't affect the test results though so I just left it there. See https://github.com/docker/docker-py/issues/1293#issuecomment-486250527 I removed the wait logic improvement due to the complexity and the fact that it triggers a bug in docker-py.

I upgraded to Xenial to ensure a newer python3-six version to solve a failure importing docker. https://github.com/docker/docker-py/issues/2294#issuecomment-486269556

With the file rename, I can now do run a subset of the tests during development:

pytest _dev/interop_test_runner.py -k SIDH -v

and add -nauto to run those tests in parallel (requires pytest-xdist). This can complete all tests in 30 seconds (with 12 jobs). Another nice feature is the ability to pause when an assertion fails and inspect the context:

pytest _dev/interop_test_runner.py -k 'test_SIDH and Server'  --pdb
kriskwiatkowski commented 5 years ago

This change is a root cause of python issuing such warn's

/usr/lib/python3.5/http/client.py:410: ResourceWarning: unclosed <socket.socket fd=8, family=AddressFamily.AF_UNIX, type=2049, proto=0, raddr=/var/run/docker.sock>
  self._close_conn()

I'm actually getting a lot of those locally. It will need to be fixed before merging

Lekensteyn commented 5 years ago

I've dropped the whole server start optimization (and kept two versions at https://github.com/cloudflare/tls-tris/commits/pwu/tests-speedup-v0 and https://github.com/cloudflare/tls-tris/commits/pwu/tests-speedup-v1).

Now it only fixes the Python 3 issues, pytest support and some potential resource leaks.