EMR configurations_json with sensible data

ivanfavi commented 4 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

terraform -version Terraform v0.12.24

provider.aws v2.56.0
provider.template v2.1.2

Affected Resource(s)

aws_emr_cluster

Terraform Configuration Files

datasource.tf

data "template_file" "emr_configuration" {
  template = file("${path.cwd}/files/configuration.json")
  vars = {
    password = data.aws_kms_secrets.emr_hive_metastore_presto.plaintext["password"]
  }
}

configurations.json

      "Classification": "hive-site",
      "Properties": {
        "javax.jdo.option.ConnectionURL": "jdbc:mysql://hive-mestatore.us-west-2.rds.amazonaws.com:3306/hive-db?createDatabaseIfNotExist=true",
        "javax.jdo.option.ConnectionDriverName": "org.mariadb.jdbc.Driver",
        "javax.jdo.option.ConnectionUserName": "hive-user",
        "javax.jdo.option.ConnectionPassword": "${password}"
      }

emr.tf

resource "aws_emr_cluster" "default" {
  other settings
  ....
  configurations_json = data.template_file.emr_configuration.rendered
  ...
}

Expected Behavior

It should keep the current status through configurations_json argument provided without expecting any change.

Plan: 0 to add, 0 to change, 0 to destroy.

------------------------------------------------------------------------

Actual Behavior

...
    ~ configurations_json               = jsonencode(
          ~ [ # forces replacement
              ~ {
                    Classification = "hive-site"
                  ~ Properties     = {
                        javax.jdo.option.ConnectionDriverName = "org.mariadb.jdbc.Driver"
                      ~ javax.jdo.option.ConnectionPassword   = "********" -> "supersecretpassword"
                        javax.jdo.option.ConnectionURL        = "jdbc:mysql://hive-mestatore.us-west-2.rds.amazonaws.com:3306/hive-db?createDatabaseIfNotExist=true"
                        javax.jdo.option.ConnectionUserName   = "hive-user"
                    } # forces replacement,
                } 
        )
...
Plan: 1 to add, 0 to change, 1 to destroy.
------------------------------------------------------------------------

Steps to Reproduce

Just perform a terraform plan or apply with sensible data in the configurations.json settings provided to the aws_emr_cluster resource

Important Factoids

If you look at the configurations in the EMR console you could see the password provided is replaced by "********" (it is encrypted in some way by amazon). I just updated the password provided by terraform for this value "********"

      "Classification": "hive-site",
      "Properties": {
        "javax.jdo.option.ConnectionURL": "jdbc:mysql://hive-mestatore.us-west-2.rds.amazonaws.com:3306/hive-db?createDatabaseIfNotExist=true",
        "javax.jdo.option.ConnectionDriverName": "org.mariadb.jdbc.Driver",
        "javax.jdo.option.ConnectionUserName": "hive-user",
        "javax.jdo.option.ConnectionPassword": "********"
      }
    },

Then terraform plan/apply didn't ask to replace the existing EMR cluster.

References

https://github.com/terraform-providers/terraform-provider-aws/issues/7733

TalkWIthKeyboard commented 3 years ago

I met the same problem, the workaround is to add a ignore_changes to the target resource.

resource "..." "..." {
  lifecycle {
    ignore_changes = [
      configurations_json
    ]
  }
}

justinretzolk commented 2 years ago

Hey @ivanfavi 👋 Thank you for taking the time to file this issue. Given that there's been a number of Terraform and AWS provider releases since you initially filed it, can you confirm whether you're still experiencing this behavior?

ivanfavi commented 2 years ago

Yes it is still happening

justinretzolk commented 2 years ago

Thank you for the update! I'm going to go ahead and label this as a bug so that we can take a look into it as time permits. Thank you again for filing this!

FoolishWiseman commented 2 years ago

This is happening in v1.1.2 today.

rnovillo commented 2 years ago

I faced the same issue today


terraform version 1.2.7
aws provider verison 4.26.0

hashicorp / terraform-provider-aws