unexpected diff on google provider update from 4.65.0 to > 5.0.0

[NemanjaMilenkovic]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to a user, that user is claiming responsibility for the issue.
• Customers working with a Google Technical Account Manager or Customer Engineer can ask them to reach out internally to expedite investigation and resolution of this issue.

Terraform Version & Provider Version(s)

cdktf 0.17.2 on

• provider registry.terraform.io/hashicorp/google v5.21.0
• provider registry.terraform.io/hashicorp/google-beta v5.21.0

Affected Resource(s)

• google_bigquery_connection_iam_member
• google_bigquery_dataset_iam_member

Terraform Configuration

  readonly editors = this.appBridgeSa.editorPrincipals.map(
    (principal) =>
      new google.bigqueryDatasetIamMember.BigqueryDatasetIamMember(
        this,
        `editor-${principal.type}-${principal.id}`,
        {
          datasetId: this.dataset.datasetId,
          role: 'roles/bigquery.dataEditor',
          member: principal,
        },
      ),
  );

Debug Output

(not relevant data redacted with "...")

 # google_bigquery_dataset_iam_member.org--editor-serviceAccount-dataform-sa_DF61663B must be replaced
-/+ resource "google_bigquery_dataset_iam_member" "org---editor-serviceAccount-dataform-sa_DF61663B" {
      + etag       = (known after apply)
      ~ id         = "projects/.../datasets/.../roles/bigquery.dataEditor/serviceAccount:service-...@gcp-sa-dataform.iam.gserviceaccount.com" -> (known after apply)
      ~ member     = "serviceAccount:service-...@gcp-sa-dataform.iam.gserviceaccount.com" # forces replacement -> (known after apply) # forces replacement
      ~ project    = "..." -> (known after apply)
        # (2 unchanged attributes hidden)

  # google_bigquery_connection_iam_member.org--dataform must be replaced
-/+ resource "google_bigquery_connection_iam_member" "org--serviceAccount-dataform" {
      ~ connection_id = "projects/.../locations/asia-northeast1/connections/custom_project_id" -> "custom_project_id"
      ~ etag          = "...=" -> (known after apply)
      ~ id            = "projects/.../locations/asia-northeast1/connections/custom_project_id/roles/bigquery.connectionUser/serviceAccount:service-...@gcp-sa-dataform.iam.gserviceaccount.com" -> (known after apply)
      ~ member        = "serviceAccount:service-...@gcp-sa-dataform.iam.gserviceaccount.com" # forces replacement -> (known after apply) # forces replacement
      ~ project       = "..." -> (known after apply)
        # (2 unchanged attributes hidden)
    }

Expected Behavior

no change

Actual Behavior

unexpeced diff on connection_id and force change on member

Steps to reproduce

1. create google_bigquery_dataset_iam_member resource
2. use google provider version < 5.0.0
3. update google provider version to >=5.0.0
4. terraform apply

Important Factoids

No response

References

No response

[ggtisc]

Hi @NemanjaMilenkovic!

I'm reviewing this scenario, but I saw that you have a lot of different versions:

• In the topic you put unexpected diff on google provider update from 4.65.0 to > 5.0.0
• In the Terraform Version & Provider Version(s) you put cdktf 0.17.2 on provider registry.terraform.io/hashicorp/google v5.21.0
• In the Steps to reproduce you put use google provider version < 5.0.0 and update google provider version to >=5.0.0

Could you please be clear on which terraform version and Google provider version are you using? And if you are using a particular terraform code for the resources google_bigquery_connection_iam_member and google_bigquery_dataset_iam_member because you are only sharing a map without more context

[NemanjaMilenkovic]

Hi @ggtisc!

• This can be seen on upgrade from Google provider major version 4 to Google provider major version 5.
• One specific example would be Google provider update from 4.65.0 to > 5.21.0.
• diff isn't created if version isn't changed. That is, if you use version 4.65.0 for example, without update, this diff wouldn't be created if no other changes are made. Also, if you only use version 5.21.0. for example, this diff wouldn't be created if no other changes are made.
• Any Terraform version should produce the same result, I am using cdktf 0.17.2

This would be an example of cdktf used that would cause it: (any google_bigquery_dataset_iam_member resource would should produce same result, there )

readonly connectionExample = new google.bigqueryConnectionIamMember.BigqueryConnectionIamMember(
  this,
  `connection-exampleId-exampleType-examplePrincipalId`,
  {
    connectionId: `projects/exampleProject/locations/exampleLocation/connections/exampleConnectionId`,
    role: 'roles/bigquery.connectionUser',
    member: `serviceAccount:${principal.email}`,
    location: 'exampleLocation',
  }
);

Terraform HCL equivalent would be something like:

resource "google_bigquery_dataset_iam_member" "user" {
  dataset_id = 'projects/exampleProject/locations/exampleLocation/connections/exampleConnectionId'
  role       =  'roles/bigquery.connectionUser'
  member     = 'serviceAccount:${principal.email}'
}

I see the cause being Terraform Data Source, used for google_bigquery_dataset_iam_member.member - this can be avoided with a workaround of using local values (indirectly referencing the managed resource values ​​through a local value). More info about data source dependencies: https://developer.hashicorp.com/terraform/language/data-sources#data-resource-dependencies

Main question here is - why is this happening on Google provider update from major version 4 to Google provider update to major version 5.

[ggtisc]

I tried to replicate this issue following your specifications, but the result was No changes. Your infrastructure matches your configuration.

The 1st thing I noticed is that you are using variables in your code like:

• map((principal)
• principal.type
• principal.id Just to mention a few. This is a good practice, but you need to have in mind that if a variable changes its value for some internal configuration it is going to result in a change.

I suggest you check your code configuration related to your variables 1 by 1 to see if any are changing its value at compilation, run or execution time.

Finally the code used to replicate this scenario is here.