Add more full featured testing of the proxy functionality

campbellC / third-wheel

A rust implementation of a man-in-the-middle proxy

MIT License

71 stars 19 forks source link

Add more full featured testing of the proxy functionality #25

Closed campbellC closed 3 years ago

campbellC commented 3 years ago

Currently we have a test that loads 'https://example.com' in chrome, firefox and curl and verifies that the content of the page is as expected. This is fine but doesn't exercise basic functionality (as seen in this issue) where it wasn't passing through query parameters.

The testing should cover all the HTTP verbs for a standard site, and should verify connection persistence (e.g. using 206 responses for large bodies).

Possible approach:

Write a test hyper TLS server (or indeed SimpleHttpServer is in the setup script) that echoes back the requests it receives in a fixed format
Write tests that run the server with a given domain name, hook up the proxy, override DNS for that domain name and runs a multitude of basic http calls.

I think it is acceptable for this to be done using curl as the basis for the verifier as an initial improvement, but ideally we would be able to override the DNS for the Chrome and Firefox tests as well.

campbellC commented 3 years ago

https://github.com/campbellC/third-wheel/commit/939b7515ff0e3292ec4e40b1e0e5cb41ed370c59

I've added the harness needed for testing third wheel. Its nearly all rust although I call out to openssl for one bit since I still haven't written the rust code to create a CA. Anyway, this harness:

Generates required certificates and a test domain
Sets up a warp TLS web server for that domain
Sets up a trivial mitm third-wheel proxy, that trusts the new certificates, and redirects calls to the domain to 127.0.0.1
A reqwest client that is set up to redirect https traffic via the third wheel proxy

I've added a single test 'foo' that verifies the whole chain of trust is working. There's plenty of room for improvement here but this means this task can just become: write a whole bunch of tests verifying behaviour that we want. The warp server and the mitm need to be more configurable to allow testing of in-stream modifications, and testing against different http sessions.

campbellC commented 3 years ago

The second set of tests I think would be ideal here would be having one client that uses the mitm and one that does not, and verifying that they see identical traffic. Likewise on the server side. Unfortunately, reqwest does not currently allow dns bypass which I believe would be required for this. I will take a swing at implementing that for reqwest and see if I can unblock this better kind of testing.

campbellC commented 3 years ago

I'm unlinking this from beta since the basic testing is up and running and will be more fleshed out as features are figured out.

campbellC commented 3 years ago

Hopefully this PR will be merged and this will unblock more full featured testing. If not, we'll need to switch clients in the test package.

campbellC commented 3 years ago

Apparently crates allows dev-dependencies to depend on github forks so there's no need to wait for the PR. I've added some tests that compare proxied and non-proxied requests and responses to verify the proxy is doing its thing. I think this suffices.