Message format in SQS - Githubissues

Hi We have quite different message format in SQS queue than expectet by this plugin (sqs/poller.rb)

Sample of our msg:

{
  "Type" : "Notification",
  "MessageId" : "60f54901-ec39-5276-1111-12922b37c1f0",
  "TopicArn" : "arn:aws:sns:us-east-1:123456789012:aws-ourdata-logs-123456789012-7ba27bdd",
  "Message" : "{\"s3Bucket\":\"<bucket_name>\",\"s3ObjectKey\":[\"AWSLogs/123456789012/OurData/eu-west-1/2022/09/08/123456789012_OurData_eu-west-1_20220908T1030Z_3QtNPaSXCXLrHaxo.json.gz\"]}",
  "Timestamp" : "2022-09-08T10:32:46.299Z",
  "SignatureVersion" : "1",
  "Signature" : "kFuAxHvSYUpdkNT0CEyXO+Mob2D7LDYHi9DDwPU56n9GvhQ53USacPVerYTsrmY2RN0MQmsp7WvGLatzLy8by7yj80v...",
  "SigningCertURL" : "https://sns.us-east-1.amazonaws.com/SimpleNotificationService-56e67fcb41f6fec09b0196692625d111.pem",
  "UnsubscribeURL" : "https://sns.us-east-1.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-east-1:123456789012:aws-ourdata-logs-564876804975-7ba27bdd:9486467e-d426-4b6b-1111-101550fe95d9"
}

While poller.rb expects probably something like:

{
  "Type" : "Notification",
  "MessageId" : "60f54901-ec39-5276-1111-12922b37c1f0",
  "TopicArn" : "arn:aws:sns:us-east-1:123456789012:aws-ourdata-logs-123456789012-7ba27bdd",
  "Records" : [
    { "eventSource": "...",
      "eventName": "...",
      "s3": { 
        "bucket": { "name": "<bucket_name>" },
        "object": { 
            "key": "AWSLogs/123456789012/ourdata/eu-west-1/2022/09/08/123456789012_OurData_eu-west-1_20220908T1030Z_3QtNPaSXCXLrHaxo.json.gz",
            "size": xxx
        }
      }
  ],
  "Timestamp" : "2022-09-08T10:32:46.299Z",
  "SignatureVersion" : "1",
  "Signature" : "kFuAxHvSYUpdkNT0CEyXO+Mob2D7LDYHi9DDwPU56n9GvhQ53USacPVerYTsrmY2RN0MQmsp7WvGLatzLy8by7yj80v...",
  "SigningCertURL" : "https://sns.us-east-1.amazonaws.com/SimpleNotificationService-56e67fcb41f6fec09b0196692625d111.pem",
  "UnsubscribeURL" : "https://sns.us-east-1.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-east-1:123456789012:aws-ourdata-logs-564876804975-7ba27bdd:9486467e-d426-4b6b-1111-101550fe95d9"
}

I created fork and made some mods - and it works (for us) with this mods. https://github.com/cherweg/logstash-input-s3-sns-sqs/compare/master...stanislav-mitrega-av:logstash-input-s3-sns-sqs:sqs_alternate_msg_format?diff=split

But I assume, we are doping something wrong and we should modify something in AWS config to get expected format of SQS notification messages. Does anybody knows what are we doing wrong?

cherweg / logstash-input-s3-sns-sqs

Message format in SQS #75