search

aws / s2n-tls

An implementation of the TLS/SSL protocols
https://aws.github.io/s2n-tls/usage-guide/
Apache License 2.0
4.52k stars 705 forks source link

Send large amounts of data through the integration test framework, with interference. #1844

Open rday opened 4 years ago

rday commented 4 years ago

Problem:

We currently test whether we can transmit data over the TLS connection, but we don't send a lot of data. We send small amounts because all the data gets displayed in stdout, which can hinder debug efforts.

We want to test sending hundreds of mbs of data. We should also try to simulate network congestion, to make sure that we don't see any corruption.

It won't be straight forward to send all this data. We may have to turn off debugging settings to make sure no data is injected by a provider. We will probably have to turn off stdout capture in the error handler (otherwise we dump a gig of data to the log if something fails, this probably won't be useful).

Possible Solutions:

Update s2nc / s2nd One possibility is to update s2nc / s2nd to store the data they receive to a file. Then we can verify the contents of this file to make sure the data was not corrupted.

Pros: This option let's us keep debugging information turned on during large data transmission. If a failure occurs, we have the entire data stream (prior to failure) stored in a file for examination.

Cons: Even more updates to s2nc/s2nd, those utils are getting complex. Storing to a file requires disk space, and storing the file. This could prevent tests that should be simple (sending a terabyte of data).

Use the process stdin/stdout PIPEs Another method is streaming data to the process' pipes. This lets us avoid storing all the data we send. We can keep a running checksum on both ends, and match when done. We should keep a counter of bytes sent/received to help debug any problems encountered.

Pros: We don't rely on disk storage. We don't need to update s2n utils.

Cons: We can't rely on stdout for debugging information (we can control s2nc/s2nd but not the other peers like openssl s_client).

rday commented 4 years ago

Take into account ideas from Matthias, https://github.com/awslabs/s2n/pull/1827.

colmmacc commented 4 years ago

Could we just use 'tee' ?

rday commented 4 years ago

We certainly can to send data. I suppose we could tee the receiving end as well. The initial output could be stripped so we know exactly which bytes to verify (this could be done in the test).

Thinking about it though, we may not need it at all. We are already working with a pipe in the test. If we keep a running checksum we don't have to worry about memory consumption.