terraform-aws-modules / terraform-aws-dms

Terraform module to create AWS DMS (Database Migration Service) resources 🇺🇦
https://registry.terraform.io/modules/terraform-aws-modules/dms/aws
Apache License 2.0
64 stars 100 forks source link

All the tasks are recreated if we add a new endpoint.  #62

Closed jogendrakumarjangid closed 6 months ago

jogendrakumarjangid commented 7 months ago

Description

We are using the module to create the DMS instance, endpoint, and tasks. if we are adding a new endpoint for a new task requirement. At the time of applying the changes, the endpoint will be create but all the current tasks also recreating. There is no change in task configuration. we are just adding a new endpoint.

Versions

Reproduction Code [Required]

Steps to reproduce the behavior:

Steps

  1. Create new dms instance, endpoint and tasks using this module
  2. Add new endpoint to configuration
  3. Try to apply the changes.

Expected behavior

The new endpoint only need to be created.

Actual behavior

New endpoint are creating but same time we can see the all the tasks are recreating.

Terminal Output Screenshot(s)

  # aws_dms_replication_task.this["track_to_s3"] must be replaced
-/+ resource "aws_dms_replication_task" "this" {
      + cdc_start_position        = (known after apply)
      ~ id                        = "development-import-mysql-tracking-to-s3" -> (known after apply)
      ~ replication_task_arn      = "arn:aws:dms:us-east-2:xxxxxxxxx:task:xyz" -> (known after apply)
      ~ replication_task_settings = jsonencode(
          ~ {
              ~ ControlTablesSettings               = {
                  - historyTimeslotInMinutes      = 5
                    # (6 unchanged attributes hidden)
                }
              ~ Logging                             = {
                  - CloudWatchLogGroup  = "dms-tasks-redshift-datasync"
                  - CloudWatchLogStream = "dms-task-syz"
                  - EnableLogContext    = false
                    # (2 unchanged attributes hidden)
                }
                # (12 unchanged attributes hidden)
            }
        )
      ~ source_endpoint_arn       = "arn:aws:dms:us-east-2:xxxxxxxxxxx:endpoint:xyz" # forces replacement -> (known after apply) # forces replacement
      ~ status                    = "running" -> (known after apply)
        tags                      = {
            "Environment" = "development"
            "Task"        = "development-import-mysql-tracking-to-s3"
            "Terraform"   = "true"
        }
      ~ target_endpoint_arn       = "arn:aws:dms:us-east-2:xxxx:endpoint:xyz" # forces replacement -> (known after apply) # forces replacement
        # (6 unchanged attributes hidden)
    }

Additional context

bryantbiggs commented 6 months ago

This is just down to the DMS API - you can see from these two lines:

~ source_endpoint_arn       = "arn:aws:dms:us-east-2:xxxxxxxxxxx:endpoint:xyz" # forces replacement -> (known after apply) # forces replacement
...
~ target_endpoint_arn       = "arn:aws:dms:us-east-2:xxxx:endpoint:xyz" # forces replacement -> (known after apply) # forces replacement

Any time you change a source or target endpoint for a task, the API requires the task to be replaced.

jogendrakumarjangid commented 6 months ago

This is just down to the DMS API - you can see from these two lines:

~ source_endpoint_arn       = "arn:aws:dms:us-east-2:xxxxxxxxxxx:endpoint:xyz" # forces replacement -> (known after apply) # forces replacement
...
~ target_endpoint_arn       = "arn:aws:dms:us-east-2:xxxx:endpoint:xyz" # forces replacement -> (known after apply) # forces replacement

Any time you change a source or target endpoint for a task, the API requires the task to be replaced.

In our case we have't make any changes to current tasks. Just added a new endpoint but still in That case all the current tasks force to replace.

elchesco commented 6 months ago

+1

juanjesus-jeeves commented 6 months ago

+1

juanjesus-jeeves commented 6 months ago

+1

This is actually also happening to me without modifying the DMS task endpoint, just by adding a new endpoint to var.endpoints

It seems like the arn of the endpoints is not known during the plan. Maybe a depends_on can solve the problem: see:

github-actions[bot] commented 5 months ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.