AWSLambdaEvents.S3.Event incorrect if event is ObjectRemoved

[mufumade]

Expected behavior

So i am processing SQS messages in aws lambda. Those messages come into my queue from s3 as a S3Notification event. Each message has a body field wich is populated with the s3 event as a JSON string.

I wanted to decode that json string to a AWSLambdaEvents.S3.Event object using the following code:

JSONDecoder().decode(AWSLambdaEvents.S3.Event.self, from: Data(self.body.utf8))

Actual behavior

expression unexpectedly raised an error: Swift.DecodingError.keyNotFound(CodingKeys(stringValue: "size", intValue: nil)

This is due to a few missing fields in the s3 event if the event is s3:ObjectRemoved:DeleteMarkerCreated or ObjectRemoved:Delete.

The following shows a real s3 object removed event:

{
  "Records": [
    {
      "eventVersion": "2.1",
      "eventSource": "aws: s3",
      "awsRegion": "eu-west-1",
      "eventTime": "2021-03-12T20: 43: 44.287Z",
      "eventName": "ObjectRemoved:DeleteMarkerCreated",
      "userIdentity": {
        "principalId": "AWS: some"
      },
      "requestParameters": {
        "sourceIPAddress": "x.x.x.x"
      },
      "responseElements": {
        "x-amz-request-id": "some",
        "x-amz-id-2": "some"
      },
      "s3": {
        "s3SchemaVersion": "1.0",
        "configurationId": "some",
        "bucket": {
          "name": "someBucketName",
          "ownerIdentity": {
            "principalId": "some"
          },
          "arn": "arn: aws: s3: : : some"
        },
        "object": {
          "key": "someKey",
          "eTag": "someTag",
          "versionId": "v.eID.MnyPlPgfmRzKDa0rhpb",
          "sequencer": "006044454BD284442DF529"
        }
      }
    }
  ]
}

Unfortunately this behavior is not documented by AWS. There is also a discussion on stackoverflow

Steps to reproduce

let deleteS3Event:String = "{\"Records\":[{\"eventVersion\":\"2.1\",\"eventSource\":\"aws:s3\",\"awsRegion\":\"eu-west-1\",\"eventTime\":\"2021-03-12T20:43:44.287Z\",\"eventName\":\"ObjectRemoved:DeleteMarkerCreated\",\"userIdentity\":{\"principalId\":\"AWS:some\"},\"requestParameters\":{\"sourceIPAddress\":\"1.1.1.1\"},\"responseElements\":{\"x-amz-request-id\":\"some\",\"x-amz-id-2\":\"some\"},\"s3\":{\"s3SchemaVersion\":\"1.0\",\"configurationId\":\"some\",\"bucket\":{\"name\":\"some\",\"ownerIdentity\":{\"principalId\":\"some\"},\"arn\":\"arn:aws:s3:::some\"},\"object\":{\"key\":\"img.png\",\"eTag\":\"some\",\"versionId\":\"some\",\"sequencer\":\"some\"}}}]}"

let s3Event:AWSLambdaEvents.S3.Event = try! JSONDecoder().decode(AWSLambdaEvents.S3.Event.self, from: Data(deleteS3Event.utf8))

This could be easily fixed by changing the size field to an optional.

//S3.swift
...
public struct Object: Codable {
        public let key: String
        public let size: UInt64?
        public let urlDecodedKey: String?
        public let versionId: String?
        public let eTag: String
        public let sequencer: String
    }
...

SwiftAWSLambdaRuntime version/commit hash

SwiftAWSLambdaRuntime version: 0.4.0

Swift & OS version (output of swift --version && uname -a)

Apple Swift version 5.3.2 (swiftlang-1200.0.45 clang-1200.0.32.28) Target: x86_64-apple-darwin20.3.0 Darwin machine_name.local 20.3.0 Darwin Kernel Version 20.3.0: Thu Jan 21 00:07:06 PST 2021; root:xnu-7195.81.3~1/RELEASE_X86_64 x86_64

[fabianfett]

@mufumade Thanks for raising this and the vast error description. Since you have already done most of the work, are you maybe interested in providing a patch pr as well? I'd be happy to accept it.

[tomerd]

@mufumade can this be closed with #195 merged?

[mufumade]

@tomerd Yes this can be closed! Thank you guys!