[Bug]: optional flag availability_zone_rebalancing results in lingering changes with refresh=false

Terraform Core Version

1.4.6

AWS Provider Version

5.77

Affected Resource(s)

aws_ecs_service

Expected Behavior

AWS provider 5.77 added support for availability_zone_rebalancing in PR 40225. The availability_zone_rebalancing is defined as an optional attribute.

I expect the aws_ecs_service configurations without this attribute defined to not generate any changes.

Actual Behavior

terraform plan -refresh=false results in the following change:

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create
  ~ update in-place

    # aws_ecs_service.service_test-name will be updated in-place
  ~ resource "aws_ecs_service" "service_test-name " {
      + availability_zone_rebalancing      = "DISABLED"

Most frustratingly, the change is not generated with regular terraform plan and terraform apply as they run with `-refresh=true. Thus the change is never reflected in the state file, either.

Relevant Error/Panic Output Snippet

No response

Terraform Configuration Files

The configuration is more or less like this (I've edited some fields for NDA reasons)

resource "aws_ecs_service" "service_test-name" {

  lifecycle {
    prevent_destroy = true
    ignore_changes  = ["task_definition", "capacity_provider_strategy", "desired_count"]
  }

  cluster                            = "test"
  name                               = "name"
  task_definition                    = "${local.fargate_dummy_task_arn}"
  desired_count                      = "1"
  deployment_minimum_healthy_percent = "100"
  deployment_maximum_percent         = "200"
  enable_ecs_managed_tags            = true
  propagate_tags                     = "SERVICE"

  capacity_provider_strategy {
    capacity_provider = "FARGATE"
    weight            = 100
  }
  platform_version = "1.4.0"

  network_configuration {
    subnets         = flatten(["${local.subnets["private_final"]}"])
    security_groups = flatten(["${local.security_groups["prod_ecs_instances"]}"])
  }

  tags = { "test:all-on-demand" = "0" }

  deployment_circuit_breaker {
    enable   = true
    rollback = true
  }

  deployment_controller {
    type = "ECS"
  }
}

Steps to Reproduce

Get an aws_ecs_service resource applied with 5.76 version
Upgrade to 5.77
tf plan -refresh=true: no changes
tf plan -refresh=false: availability_zone_rebalancing = DISABLED added to the plan output despite being optional

Debug Output

No response

Panic Output

No response

Important Factoids

We have thousands of terraform projects; we use refresh=false during the plan in some of our CI/CD steps to avoid being rate-limited. We cannot simply add this optional attribute to all of our configurations, as we, regrettably, have a large number of projects still using older AWS provider versions.

References

No response

Would you like to implement a fix?

None

hashicorp / terraform-provider-aws