integration testsuite

[tintinweb]

created a basic integration testsuite

• manages external tls server (via Popen)
• implements python tls server (via multiprocessing.Process)
• implements example full_rsa_connection_with_application_data.py
• based on unittest to have a common test interface
• moved to a different folder to separate the tests from unittests

todo:

• add tests for sniffer and other examples
• move common stuff to own module to have a clean testsuite
• fix bugs
• add external tests (check tls compatibility with public webservices)

Note: this is based on master without changes introduced with #77

[alexmgr]

Had something different in mind. Was thinking about something to test the examples themselves. Maybe have a subprocess which spawns the examples against the various servers, and assert on the return code?

Other then that, I still think this is useful in testing the various client side scenarios. Maybe do_test should take an optional extensions parameter?

[tintinweb]

Finally had some time to continue the integration suite. I started with the external tests (api vs real server/client) just to check if my process wrapper classes work fine :) Feel free to have a quick look at the recently pushed examples testsuite. The code is pretty ugly but it works for now and is surprisingly fast.

A few testcases are still failing. I'll have them fixed/implemented in the next days :bowtie:

======================================================================
ERROR: test_security_scanner_sniffer_mode (__main__.TestExampleSnifferWithOpenSslServerAndClient)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_examples.py", line 148, in test_security_scanner_sniffer_mode
    raise NotImplementedError("NOT YET IMPLEMENTED")
NotImplementedError: NOT YET IMPLEMENTED

======================================================================
ERROR: test_sessionctx_sniffer_lo (__main__.TestExampleSnifferWithOpenSslServerAndClient)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_examples.py", line 152, in test_sessionctx_sniffer_lo
    raise NotImplementedError("NOT YET IMPLEMENTED")
NotImplementedError: NOT YET IMPLEMENTED

======================================================================
FAIL: test_client_hello_twice_py (__main__.TestExampleClientsAgainstLocalOpenSsl)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_examples.py", line 83, in test
    self.assertEqual(pid.getReturnCode(), 0)
AssertionError: 1 != 0

======================================================================
FAIL: test_client_rsa_mutual_auth_py (__main__.TestExampleClientsAgainstLocalOpenSsl)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_examples.py", line 83, in test
    self.assertEqual(pid.getReturnCode(), 0)
AssertionError: 1 != 0

======================================================================
FAIL: test_sessionctx_sniffer_pcap_mode (__main__.TestExampleClientsAgainstLocalOpenSsl)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_examples.py", line 71, in test_sessionctx_sniffer_pcap_mode
    self.assertEqual(pid.getReturnCode(), 0)
AssertionError: 1 != 0

----------------------------------------------------------------------
Ran 19 tests in 146.940s

FAILED (failures=3, errors=2)

[tintinweb]

debugging travis-ci instances is such a pain without having access to a public ip hosted reverse shell listener. Anyway, turns out their container runs an openssl from 2012 that kind of freaks out on ECDHE-DSS (one of the integration_external tests) but I finally managed to make the integration suites work by pinning the travis-ci instance distribution to an experimental ubuntu 14.04.

Also made the test_suites run in parallel and disabled incomplete and always failing tests. I've created #84 to track them.

• unittests
• integration_external - runs scapy-ssl_tls against a local openssl
• integration_examples - runs selected examples

lets see how reliable the integration suite results are and decide whether to leave them in or go back to manual execution :)

[tintinweb]

conflicts with #85 and its probably easier to first finish #85 and then merge this. what do you think @alexmgr ? otherwise it'd be ready to merge.

[alexmgr]

Hi @tintinweb,

Sorry, running late on this one. I'll try and check this out this week end or next week. Has a cursory look and it's looking good. Would have been great to have while doing the work on TLS 1.3 :).

Yes, if we could rebase this on top of #85 would definitely make life easier for me ;). But #85 might stay unmerged for a while, cause porting the various examples might take some time.

Cheers, Alex

[tintinweb]

Hi @alexmgr,

It would probably make sense to merge #78 to master since #85 might stay open for a bit longer and then rebase #85 off master. This should only lead to minor conflicts in some example scripts. Is #85 on github in sync with your local changes? Just asking because if I rebase + force push it your local branch is going to conflict.

[alexmgr]

Sure go ahead.

My local branch for #85 is not really in sync, but there is nothing interesting there. In short it's fine ;)