simplifying download/upload logic

airbnb / binaryalert

BinaryAlert: Serverless, Real-time & Retroactive Malware Detection.

https://binaryalert.io

Apache License 2.0

1.41k stars 187 forks source link

simplifying download/upload logic #147

Closed ryandeivert closed 5 years ago

ryandeivert commented 5 years ago

to @Ryxias cc: @airbnb/binaryalert-maintainers

Background

The Carbon Black download logic was more complicated than necessary. The Binary model provides a file-like-object as file that can be read from directly.

Changes

Updating the download/upload logic to use a streaming file read/write to perform a direct upload to s3.
This removes the need to store any data locally, also removing the need to "shred" the data.

Testing

Update to unit test to check new logic.

coveralls commented 5 years ago

Coverage decreased (-0.1%) to 91.111% when pulling d5c71e2229d66a5c510f2f42adf05377bd36b124 on ryandeivert-update-download-logic into 754a51086ca9b0a2b250292dc8c3844c908d76b6 on master.