brexhq / substation

Substation is a toolkit for routing, normalizing, and enriching security event and audit logs.
https://substation.readme.io
MIT License
330 stars 21 forks source link

S3 object key url encoding issue #97

Closed Bin-security closed 1 year ago

Bin-security commented 1 year ago

Describe the bug I am writing to report a bug in the s3 sns handler in substation. The function call at line tries to download an object with a bucket name and object key. The object key in the SNS notification event is url encoded, according to this document. The key needs to be url-decoded before downloading the object. Otherwise, Substation reports the error "NoSuchKey: The specified key does not exist".

Error Message s3 sns handler: s3manager download bucket my-bucket key my-prefix/date%3D2023-04-14/hour%3D00/my-object.gz: NoSuchKey: The specified key does not exist.\n\tstatus code: 404.

The object exists in the bucket. The key is my-prefix/date=2023-04-14/hour=00/my-object.gz

Expected behavior Substation should be able to download the object which exists in the bucket.

Cloud (please complete the following information):

jshlbrd commented 1 year ago

Thanks @Bin-security, I also confirmed this bug and merged a fix into main from #96

Bin-security commented 1 year ago

Thanks @jshlbrd for the quick fix.

jshlbrd commented 1 year ago

No problem, thanks for raising it!