airbnb / binaryalert

BinaryAlert: Serverless, Real-time & Retroactive Malware Detection.
https://binaryalert.io
Apache License 2.0
1.4k stars 187 forks source link

Create a different queue for retroactive analysis #134

Open austinbyers opened 5 years ago

austinbyers commented 5 years ago

Background

Related to #102 , it's always felt awkward to have the same SQS queue for both regular operation and for retroactive analysis:

These problems could all be fixed if there were just a different SQS queue for retroactive analysis!

Desired Change

Create a new SQS queue specifically for retro scans. It's not clear whether Lambda will allow 2 different SQS event source mappings, but it's worth a shot.

jdheyburn commented 3 years ago

On top of this, might it be helpful to publish retrospective no_yara_matches to a different SNS topic?

Background behind that is, users might have a flow whereby they subscribe to no_yara_match for continued file processing. If retrospective scanning were to publish messages on that topic once again, they could be double-processed.