Unusual behaviour when downloading partial chunks

[traktion]

I've enabled streaming in my sn_httpd client using the files API, but I uncovered some unusual behaviour.

The associated code is here: https://github.com/traktion/sn_httpd/blob/163e59181502e6783b71027ed55328ad7f5920f1/src/main.rs#L170

When using STREAM_CHUNK_SIZE smaller than a safe net chunk size (up to 512 KB, I believe), there seems to be unexpected overlapping data returned.

For example, when using 131072 as the STREAM_CHUNK_SIZE, one of my test images was being corrupted. On closer inspection with Wireshark, it looked like two bytes were being repeated on the subsequent call.

I took the output from Wireshark and compared it. I've attached a screenshot of the difference:

When using much smaller STREAM_CHUNK_SIZE, it exacerbates the issue (although performance suffers regardless).

I can't see any issues in my code and it looks like position is being measured / applied incorrectly somewhere within this code block (inc. dependencies): https://github.com/maidsafe/safe_network/blob/e92b8e46bb332daa4aeddf1ec255704cd977b99e/sn_client/src/files/download.rs#L156

I haven't seen any issues when downloading with position == 0 and MAX length, but for very large files, it's inefficient for streaming using that technique.

[JasonPaulGithub]

Amazing work @traktion I will cc @RolandSherwin about this and mark the issue as a bug for now. Thanks!

[traktion]

From digging more into this, I tracked down the issue to self_encryption and I have a fix for it. I'll create a PR on that repo instead and we can close this one out if/when it's confirmed.