hashicorp / terraform-provider-aws

The AWS Provider enables Terraform to manage AWS resources.
https://registry.terraform.io/providers/hashicorp/aws
Mozilla Public License 2.0
9.73k stars 9.09k forks source link

[Bug]: aws_datasync_task continuous drift on simple config #36016

Open Gowiem opened 6 months ago

Gowiem commented 6 months ago

Terraform Core Version

tofu v1.6.1

AWS Provider Version

v5.38.0

Affected Resource(s)

Expected Behavior

When no changes to configuration, no changes should show up in the plan.

Actual Behavior

On each plan of a simple aws_datasync_task resource, we're getting recurring "will be updated-in-place" changes on the task_report_config.report_overrides attribute block.

Relevant Error/Panic Output Snippet

OpenTofu will perform the following actions:

  # module.datasync_task.aws_datasync_task.default will be updated in-place
  ~ resource "aws_datasync_task" "default" {
        id                       = "arn:aws:datasync:us-east-2:123412341234:task/task-0000000000000"
        name                     = "mp-dev-datasync-example"
        tags                     = {
            "Name"      = "mp-dev-datasync-example"
            "Namespace" = "mp"
            "Stage"     = "dev"
        }
        # (4 unchanged attributes hidden)

      ~ task_report_config {
            # (3 unchanged attributes hidden)

          - report_overrides {}

            # (1 unchanged block hidden)
        }

        # (1 unchanged block hidden)
    }

Terraform Configuration Files

Will post a child module that reproduces the issue soon.

# main.tf
resource "aws_datasync_task" "default" {
  name = module.this.id

  source_location_arn      = aws_datasync_location_nfs.source.arn
  destination_location_arn = aws_datasync_location_s3.destination.arn

  options {
    bytes_per_second = var.task_bytes_per_second
  }

  schedule {
    schedule_expression = var.task_schedule_expression
  }

  task_report_config {
    s3_object_versioning = var.task_report_s3_object_versioning
    output_type          = var.task_report_output_type
    report_level         = var.task_report_level

    s3_destination {
      s3_bucket_arn          = data.aws_s3_bucket.selected.arn
      subdirectory           = "datasync-reports/"
      bucket_access_role_arn = module.s3_access_role.arn
    }
  }

  tags = module.this.tags
}

## Locations
#############

resource "aws_datasync_location_nfs" "source" {
  server_hostname = var.nfs_server_hostname
  subdirectory    = var.nfs_subdirectory

  on_prem_config {
    agent_arns = [var.datasync_agent_arn]
  }

  mount_options {
    version = var.nfs_mount_version
  }

  tags = module.this.tags
}

resource "aws_datasync_location_s3" "destination" {
  s3_bucket_arn = data.aws_s3_bucket.selected.arn
  subdirectory  = var.s3_subdirectory

  s3_config {
    bucket_access_role_arn = module.s3_access_role.arn
  }

  tags = module.this.tags
}

module "s3_access_role" {
  source  = "cloudposse/iam-role/aws"
  version = "0.19.0"

  attributes = ["s3-access"]

  policy_description = "Allow DataSync S3 Access"
  role_description   = "IAM role with permissions to allow DataSync to perform actions on S3 the given bucket"

  principals = {
    Service = ["datasync.amazonaws.com"]
  }

  policy_documents = [
    data.aws_iam_policy_document.datasync_s3_access_policy.json,
  ]

  context = module.this.context
}

data "aws_iam_policy_document" "datasync_s3_access_policy" {
  statement {
    actions = [
      "s3:ListBucket",
      "s3:GetBucketLocation",
      "s3:ListBucketMultipartUploads",
    ]
    effect = "Allow"
    resources = [
      data.aws_s3_bucket.selected.arn
    ]
  }

  statement {
    actions = [
      "s3:AbortMultipartUpload",
      "s3:DeleteObject",
      "s3:GetObject",
      "s3:GetObjectTagging",
      "s3:GetObjectVersion",
      "s3:GetObjectVersionTagging",
      "s3:ListMultipartUploadParts",
      "s3:PutObject",
      "s3:PutObjectTagging",
    ]
    effect = "Allow"
    resources = [
      "${data.aws_s3_bucket.selected.arn}/*"
    ]
  }
}

Steps to Reproduce

  1. Create a root module that uses aws_datasync_task.
  2. Cleanly apply it
  3. Plan after the apply
  4. See that the configuration has drifted with 1 change like above plan output.

Debug Output

No response

Panic Output

No response

Important Factoids

No response

References

No response

Would you like to implement a fix?

None

github-actions[bot] commented 6 months ago

Community Note

Voting for Prioritization

Volunteering to Work on This Issue

justinretzolk commented 6 months ago

Hey @Gowiem 👋 Thank you for taking the time to raise this! Can you verify whether the issue persists using Terraform as well? Debug logs are often useful as well, if you're able to provide them (redacted as needed).

elvis-mofor commented 2 months ago

@Gowiem Were you able to get a fix for this issue?

Gowiem commented 2 months ago

@elvis-mofor Nope. I believe this is still occurring and creating drift in our automation -- we just auto-apply over it. I don't have an easy way to confirm the same issue with Terraform. @oycyc might be able to provide some input as he's picked up my work on DMS.

oycyc commented 2 months ago

Hey @Gowiem 👋 Thank you for taking the time to raise this! Can you verify whether the issue persists using Terraform as well? Debug logs are often useful as well, if you're able to provide them (redacted as needed).

@Gowiem Were you able to get a fix for this issue?

@justinretzolk @elvis-mofor This is still a persisting issue. Every time a plan / apply happens, these attributes changes in place even though there's nothing changing:

Screenshot 2024-07-06 at 3 23 32 PM

See these debug logs requested (with arn and other info removed): https://gist.github.com/oycyc/18daa8c0fbe280e191ef4585ed01e0e8