S3 target iam policy not following user guide

[raphavr]

Description

The S3 target policy configured by the module is not following the AWS doc, hence leading to permission errors.

• [X] ✋ I have searched the open/closed issues and my issue is not listed.

Versions

• Module version [Required]:

• Terraform version: 1.5.7

• Provider version(s): 5.48.0

Reproduction Code [Required]

s3_endpoints = {
    s3-target = {
        endpoint_id   = "cdc-test-target"
        endpoint_type = "target"
        engine_name   = "s3"

        bucket_folder               = "source=dms"
        bucket_name                 = "raw.cdc.test.raphavr.tools"
        compression_type            = "GZIP"
        data_format                 = "parquet"
        date_partition_enabled      = true
        date_partition_delimiter    = "none"
        ssl_mode                    = "none"
        tags                        = {}
    }
}

access_target_s3_bucket_arns = ["arn:aws:s3:::raw.cdc.test.raphavr.tools"]

Expected behavior

A policy as below should be created:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "s3:PutObjectTagging",
                "s3:PutObject",
                "s3:DeleteObject"
            ],
            "Effect": "Allow",
            "Resource": "arn:aws:s3:::raw.cdc.test.raphavr.tools/*",
            "Sid": "S3ObjectsTarget"
        },
        {
            "Action": [
                "s3:ListBucket"
            ],
            "Effect": "Allow",
            "Resource": "arn:aws:s3:::raw.cdc.test.raphavr.tools",
            "Sid": "S3BucketTarget"
        }
    ]
}

Actual behavior

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "s3:PutObjectTagging",
                "s3:PutObject",
                "s3:DeleteObject",
                "s3:ListBucket"
            ],
            "Effect": "Allow",
            "Resource": "arn:aws:s3:::raw.cdc.test.raphavr.tools",
            "Sid": "S3Target"
        }
    ]
}

Additional context

I've opened a PR following the user guide

[bryantbiggs]

can you show the error message you are getting?

[sigfigsteve]

I'm experiencing the same issue. The first error I get is in cloudwatch logs when I start with a full load and it tries to truncate existing data from the target bucket. It shows up in cloudwatch logs: 2024-06-10T17:19:14 [TARGET_LOAD ]E: Failed to delete s3 object schema/table/LOAD00000001.parquet [1021807] (file_imp.c:2758) This happens despite the endpoint test from the AWS console succeeding. AWS docs at https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Target.S3.html show the correct permissions that @raphavr is setting. When I add a /* resource identifier for my bucket policy, it works for me too.