SQS not deleteing - Githubissues

We pulled your latest changes and one of the developers is haveing an issue with the Que:

Background

We have encountered some issue with the latest update of the Binary Alert analyzer function.

The behavior of this issue is we are getting duplicated scanning notification every 5min. To reproduce the issue, you can deploy the v0.6 Binary Alert terraform module. Subscribe to the SNS topic with test email, and upload a file to the S3 bucket, this will cause a new message added in SQS queue. After file being analyzed, an SNS notification is published to the topic. And after every 5min, the SQS triggers the lambda function again even when there are no new files uploaded to s3, and I got the notification from SNS with the same file.

Seems the issue is the analyzer lambda function could not delete the analyzed SQS message. This is what we have in the lambda log:

[INFO] 2018-07-12T13:57:14.523Z 759fa9ea-85db-11e8-8dd9-2b2990818c65 Analyzing "test-k-analyser-bucket:google day.docx" [INFO] 2018-07-12T13:57:14.527Z 759fa9ea-85db-11e8-8dd9-2b2990818c65 Resetting dropped connection: test-k-analyser-bucket.s3.eu-west-1.amazonaws.com [INFO] 2018-07-12T13:57:15.58Z 759fa9ea-85db-11e8-8dd9-2b2990818c65 S3:test-k-analyser-bucket:google day.docx did not match any YARA rules [INFO] 2018-07-12T13:57:15.59Z 759fa9ea-85db-11e8-8dd9-2b2990818c65 Publishing an SNS alert [INFO] 2018-07-12T13:57:15.87Z 759fa9ea-85db-11e8-8dd9-2b2990818c65 Starting new HTTPS connection (1): monitoring.eu-west-1.amazonaws.com END RequestId: 759fa9ea-85db-11e8-8dd9-2b2990818c65 REPORT RequestId: 759fa9ea-85db-11e8-8dd9-2b2990818c65 Duration: 9232.94 ms Billed Duration: 9300 ms Memory Size: 1024 MB Max Memory Used: 316 MB

This might happen because of the “receipts_to_delete” condition added with the latest change from the main.py file:

# Delete all of the SQS receipts (mark them as completed).
receipts_to_delete = [msg['receipt'] for msg in event.get('messages', [])]
if receipts_to_delete:
    analyzer_aws_lib.delete_sqs_messages(event['queue_url'], receipts_to_delete)

I have now rolled back to the previous version of the analyzer lambda function and the SQS cleanup works as expected now. Can you please help look into this issue? Thanks very much.

airbnb / binaryalert

SQS not deleteing #126

Background