search

hashicorp / terraform-provider-aws

The AWS Provider enables Terraform to manage AWS resources.
https://registry.terraform.io/providers/hashicorp/aws
Mozilla Public License 2.0
9.81k stars 9.15k forks source link

Synthetics Canary update not working #17642

Closed aws-user-2021 closed 3 years ago

aws-user-2021 commented 3 years ago

Terraform CLI and Terraform AWS Provider Version

Terraform v0.14.6

Affected Resource(s)

aws_synthetics_canary

Terraform Configuration Files

  ~ resource "aws_synthetics_canary" "synthetics" {
        id                       = "cm-dev-testui"
        name                     = "cm-dev-testui"
      ~ zip_file                 = "files/testui-canary-298c366a80006e33f3b97d2dbc9a4c8f.zip" -> "files/testui-canary-082d98273331513b8d3c4ae575c3fbb2.zip"
        # (12 unchanged attributes hidden)

        # (2 unchanged blocks hidden)
    }

Debug Output

2021-02-16T13:27:43.115-0500 [INFO] plugin.terraform-provider-aws_v3.28.0_x5: 2021/02/16 13:27:43 [DEBUG] [aws-sdk-go] DEBUG: Request synthetics/GetCanary Details:

State is RUNNING

2021-02-16T13:27:47.372-0500 [INFO] plugin.terraform-provider-aws_v3.28.0_x5: 2021/02/16 13:27:47 [DEBUG] [aws-sdk-go] DEBUG: Request synthetics/StopCanary Details:

2021-02-16T13:27:47.590-0500 [INFO] plugin.terraform-provider-aws_v3.28.0_x5: 2021/02/16 13:27:47 [DEBUG] [aws-sdk-go] DEBUG: Request synthetics/GetCanary Details:

State is STOPPING

2021-02-16T13:27:48.610-0500 [INFO] plugin.terraform-provider-aws_v3.28.0_x5: 2021/02/16 13:27:48 [DEBUG] [aws-sdk-go] DEBUG: Request synthetics/GetCanary Details:

State is STOPPED

2021-02-16T13:27:48.795-0500 [INFO] plugin.terraform-provider-aws_v3.28.0_x5: 2021/02/16 13:27:48 [DEBUG] [aws-sdk-go] DEBUG: Request synthetics/UpdateCanary Details:

State is UPDATING

2021-02-16T13:28:13.377-0500 [INFO] plugin.terraform-provider-aws_v3.28.0_x5: 2021/02/16 13:28:13 [DEBUG] [aws-sdk-go] DEBUG: Request synthetics/GetCanary Details:

State is STOPPED

2021/02/16 13:28:13 [DEBUG] aws_synthetics_canary.synthetics[0]: apply errored, but we're indicating that via the Error pointer rather than returning it: error waiting for Synthetics Canary (cm-dev-testui) updating: unexpected state 'STOPPED', wanted target 'READY'. last error: %!s()

Error: error waiting for Synthetics Canary (cm-dev-testui) updating: unexpected state 'STOPPED', wanted target 'READY'. last error: %!s()

Expected Behavior

canary should have been restarted but instead terraform update fails.

Actual Behavior

looks like the code is expecting the canary to be in READY state after update but CloudWatch Synthetics is putting the canary backed to STOPPED state since it was in STOPPED state before UpdateCanary API was called. It is likely that AWS updated behavior after the developer had tested updating canary scenario.

Steps to Reproduce

Update the canary code and run terraform apply

aws-user-2021 commented 3 years ago

Here is some more info. When a canary is first created and not started, it goes into READY state (corresponding AWS Console state is "Not started"). If the canary code is updated while in READY state, the state changes to UPDATING while update is in progress and then changes back to READY state once update is done.

However, once the canary has been started, the update process stops the canary first (state changes to STOPPING and then to STOPPED) before updating the code. The state change in this case goes from STOPPED to UPDATING and then back to STOPPED and not READY. Current implementation is expecting the canary state to go to READY after the code update completes and hence canary update fails in this scenario, which is the most common use case.

aws-user-2021 commented 3 years ago

One more issue canary resource replacement works but when you try to delete a running canary, it fails. AWS requires that canary be stopped before it is deleted. Looks like the code to stop the canary before deletion is not working.

aws-user-2021 commented 3 years ago

Primary issue mentioned regarding update failing when code is updated also happens when tags are updated likely because of the same reason.

bill-rich commented 3 years ago

Hi @aws-user-2021! Can you please provide your Terraform configuration to help reproduce the issue?

DrFaust92 commented 3 years ago

Im able to reproduce the issue, ill take a crack at fixing it.

aws-user-2021 commented 3 years ago

@DrFaust92 thank you. Deleting canary is also failing and I am getting by right now using a destroy provisioner to stop the canary. Not sure if you were able to reproduce this issue.

ghost commented 3 years ago

This has been released in version 3.30.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

ghost commented 3 years ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!