ProviderMeta Issue enrolling GCP resource into PlanResourceChange

VenelinMartinov commented 4 months ago

What happened?

While working on https://github.com/pulumi/pulumi-gcp/issues/1874 I attempted to enroll the resource into PlanResourceChange in order to address the diff issue.

Diffing fails with the following error:

--- FAIL: TestConnProfileUpgradePermaDiff (0.21s)
    replay.go:204:
            Error Trace:    /Users/vvm/go/pkg/mod/github.com/pulumi/providertest@v0.0.11/replay/replay.go:204
                                        /Users/vvm/go/pkg/mod/github.com/pulumi/providertest@v0.0.11/replay/replay.go:186
                                        /Users/vvm/go/pkg/mod/github.com/pulumi/providertest@v0.0.11/replay/replay.go:121
                                        /Users/vvm/go/pkg/mod/github.com/pulumi/providertest@v0.0.11/replay/replay.go:169
                                        /Users/vvm/code/pulumi-gcp/provider/provider_yaml_test.go:846
            Error:          Received unexpected error:
                            ProviderMeta: recoverCtyValue failed: Cannot recover cty.Value
                              Desired type: cty.Object(map[string]cty.Type{"module_name":cty.String})
                              Value: &{{        }    [] pulumi-development     [https://www.googleapis.com/auth/cloud-platform https://www.googleapis.com/auth/userinfo.email] 0x140026b77e0 false  0s map[] false  10s 0x140026d1650 context.Background Pulumi/3.0 (GPN:Pulumi; https://www.pulumi.com) pulumi-gcp/dev [{0x14002908088} {0x14002908090}] 0x140028380a0 https://accessapproval.googleapis.com/v1/ https://accesscontextmanager.googleapis.com/v1/ https://managedidentities.googleapis.com/v1beta1/ https://alloydb.googleapis.com/v1beta/ https://apigateway.googleapis.com/v1beta/ https://apigee.googleapis.com/v1/ https://appengine.googleapis.com/v1/ https://apphub.googleapis.com/v1/ https://artifactregistry.googleapis.com/v1/ https://backupdr.googleapis.com/v1/ https://beyondcorp.googleapis.com/v1/ https://biglake.googleapis.com/v1/ https://bigquery.googleapis.com/bigquery/v2/ https://analyticshub.googleapis.com/v1beta1/ https://bigqueryconnection.googleapis.com/v1/ https://bigquerydatapolicy.googleapis.com/v1/ https://bigquerydatatransfer.googleapis.com/v1/ https://bigqueryreservation.googleapis.com/v1/ https://bigtableadmin.googleapis.com/v2/ https://billingbudgets.googleapis.com/v1/ https://binaryauthorization.googleapis.com/v1/ https://blockchainnodeengine.googleapis.com/v1/ https://certificatemanager.googleapis.com/v1/ https://cloudasset.googleapis.com/v1/ https://cloudbuild.googleapis.com/v1/ https://cloudbuild.googleapis.com/v2/ https://clouddeploy.googleapis.com/v1/ https://domains.googleapis.com/v1beta1/ https://cloudfunctions.googleapis.com/v1/ https://cloudfunctions.googleapis.com/v2beta/ https://cloudidentity.googleapis.com/v1beta1/ https://ids.googleapis.com/v1/ https://cloudquotas.googleapis.com/v1/ https://{{location}}-run.googleapis.com/ https://run.googleapis.com/v2/ https://cloudscheduler.googleapis.com/v1/ https://cloudtasks.googleapis.com/v2/ https://compute.googleapis.com/compute/beta/ https://containeranalysis.googleapis.com/v1beta1/ https://{{location}}-gkemulticloud.googleapis.com/v1/ https://cloudbilling.googleapis.com/v1/ https://datamigration.googleapis.com/v1/ https://datacatalog.googleapis.com/v1beta1/ https://dataform.googleapis.com/v1beta1/ https://datafusion.googleapis.com/v1beta1/ https://dlp.googleapis.com/v2/ https://datapipelines.googleapis.com/v1/ https://dataplex.googleapis.com/v1/ https://dataproc.googleapis.com/v1beta2/ https://metastore.googleapis.com/v1beta/ https://datastore.googleapis.com/v1/ https://datastream.googleapis.com/v1/ https://www.googleapis.com/deploymentmanager/v2/ https://dialogflow.googleapis.com/v2/ https://{{location}}-dialogflow.googleapis.com/v3/ https://{{location}}-discoveryengine.googleapis.com/v1/ https://dns.googleapis.com/dns/v1beta2/ https://{{location}}-documentai.googleapis.com/v1/ https://contentwarehouse.googleapis.com/v1/ https://edgecontainer.googleapis.com/v1/ https://edgenetwork.googleapis.com/v1/ https://essentialcontacts.googleapis.com/v1/ https://file.googleapis.com/v1beta1/ https://firebase.googleapis.com/v1beta1/ https://firebaseappcheck.googleapis.com/v1beta/ https://firebasedatabase.googleapis.com/v1beta/ https://firebaseextensions.googleapis.com/v1beta/ https://firebasehosting.googleapis.com/v1beta1/ https://firebasestorage.googleapis.com/v1beta/ https://firestore.googleapis.com/v1/ https://gkebackup.googleapis.com/v1/ https://gkehub.googleapis.com/v1beta1/ https://gkehub.googleapis.com/v1beta/ https://gkeonprem.googleapis.com/v1/ https://healthcare.googleapis.com/v1beta1/ https://iam.googleapis.com/v2beta/ https://iam.googleapis.com/v1/ https://iam.googleapis.com/v1/ https://iap.googleapis.com/v1/ https://identitytoolkit.googleapis.com/v2/ https://connectors.googleapis.com/v1/ https://cloudkms.googleapis.com/v1/ https://logging.googleapis.com/v2/ https://looker.googleapis.com/v1/ https://memcache.googleapis.com/v1beta2/ https://migrationcenter.googleapis.com/v1/ https://ml.googleapis.com/v1/ https://monitoring.googleapis.com/ https://netapp.googleapis.com/v1beta1/ https://networkconnectivity.googleapis.com/v1/ https://networkmanagement.googleapis.com/v1/ https://networksecurity.googleapis.com/v1beta1/ https://networkservices.googleapis.com/v1/ https://notebooks.googleapis.com/v1/ https://orgpolicy.googleapis.com/v2/ https://osconfig.googleapis.com/v1beta/ https://oslogin.googleapis.com/v1/ https://privateca.googleapis.com/v1/ https://publicca.googleapis.com/v1beta1/ https://pubsub.googleapis.com/v1/ https://{{region}}-pubsublite.googleapis.com/v1/admin/ https://redis.googleapis.com/v1beta1/ https://cloudresourcemanager.googleapis.com/v1/ https://runtimeconfig.googleapis.com/v1beta1/ https://secretmanager.googleapis.com/v1/ https://securesourcemanager.googleapis.com/v1/ https://securitycenter.googleapis.com/v1/ https://securityposture.googleapis.com/v1/ https://websecurityscanner.googleapis.com/v1beta/ https://servicedirectory.googleapis.com/v1beta1/ https://servicemanagement.googleapis.com/v1/ https://serviceusage.googleapis.com/v1beta1/ https://sourcerepo.googleapis.com/v1/ https://spanner.googleapis.com/v1/ https://sqladmin.googleapis.com/sql/v1beta4/ https://storage.googleapis.com/storage/v1/ https://storageinsights.googleapis.com/v1/ https://storagetransfer.googleapis.com/v1/ https://cloudresourcemanager.googleapis.com/v3/ https://tpu.googleapis.com/v1/ https://tpu.googleapis.com/v2/ https://{{region}}-aiplatform.googleapis.com/v1beta1/ https://vmwareengine.googleapis.com/v1/ https://vpcaccess.googleapis.com/v1beta1/ https://notebooks.googleapis.com/v2/ https://workflows.googleapis.com/v1/ https://workstations.googleapis.com/v1beta/ https://cloudbilling.googleapis.com/v1/ https://composer.googleapis.com/v1beta1/ https://container.googleapis.com/v1beta1/ https://dataflow.googleapis.com/v1b3/ https://iamcredentials.googleapis.com/v1/ https://cloudresourcemanager.googleapis.com/v3/ https://iam.googleapis.com/v1/  https://servicenetworking.googleapis.com/v1/ https://bigtableadmin.googleapis.com/v2/ https://{{location}}-cloudresourcemanager.googleapis.com/v3/ https://{{location}}-gkemulticloud.googleapis.com/v1/ https://{{location}}-gkemulticloud.googleapis.com/v1/ 0x14002680f00 0x14002680f40}
                              Actual type: *transport.Config

                            diffing urn:pulumi:dev::connection-profile::gcp:datastream/connectionProfile:ConnectionProfile::dbReplicationConnectionProfile
                            github.com/pulumi/pulumi-terraform-bridge/v3/pkg/tfbridge.(*Provider).Diff
                                /Users/vvm/code/pulumi-terraform-bridge/pkg/tfbridge/provider.go:896
                            github.com/pulumi/pulumi-terraform-bridge/x/muxer.(*muxer).Diff.func1
                                /Users/vvm/code/pulumi-terraform-bridge/x/muxer/muxer.go:369
                            github.com/pulumi/pulumi-terraform-bridge/x/muxer.resourceMethod[...]
                                /Users/vvm/code/pulumi-terraform-bridge/x/muxer/muxer.go:334
                            github.com/pulumi/pulumi-terraform-bridge/x/muxer.(*muxer).Diff
                                /Users/vvm/code/pulumi-terraform-bridge/x/muxer/muxer.go:368
                            github.com/pulumi/providertest/replay.replay[...]
                                /Users/vvm/go/pkg/mod/github.com/pulumi/providertest@v0.0.11/replay/replay.go:185
                            github.com/pulumi/providertest/replay.Replay
                                /Users/vvm/go/pkg/mod/github.com/pulumi/providertest@v0.0.11/replay/replay.go:121
                            github.com/pulumi/providertest/replay.ReplaySequence
                                /Users/vvm/go/pkg/mod/github.com/pulumi/providertest@v0.0.11/replay/replay.go:169
                            github.com/pulumi/pulumi-gcp/provider/v7.TestConnProfileUpgradePermaDiff
                                /Users/vvm/code/pulumi-gcp/provider/provider_yaml_test.go:846
                            testing.tRunner
                                /opt/homebrew/Cellar/go/1.22.1/libexec/src/testing/testing.go:1689
                            runtime.goexit
                                /opt/homebrew/Cellar/go/1.22.1/libexec/src/runtime/asm_arm64.s:1222
            Test:           TestConnProfileUpgradePermaDiff
FAIL
exit status 1
FAIL    github.com/pulumi/pulumi-gcp/provider/v7    1.395s

Example

Running TestConnProfileUpgradePermaDiff in https://github.com/pulumi/pulumi-gcp/pull/1890

Output of `pulumi about`

bridge commit 282c9dd7f749bba2987f8b5cbe8a72c5bab4c3ed gcp commit 768e01e

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

VenelinMartinov commented 4 months ago

The issue happens here: https://github.com/pulumi/pulumi-terraform-bridge/blob/f6013b69e741bc73632fd42ba2b4630a1f2cc3ff/pkg/tfshim/sdk-v2/provider2.go#L280-L295

Looks like we try to coerce the GCP transport.Config value into a cty.Value and fail.

The GCP config is defined here: https://github.com/hashicorp/terraform-provider-google-beta/blob/99afea8acf81d576b882ec7b4418979139a00e1a/google-beta/transport/config.go#L161

I believe this bit is likely untested since AWS does not define a providerMeta at all.

VenelinMartinov commented 4 months ago

PlanResourceChange does not seem to help with https://github.com/pulumi/pulumi-gcp/issues/1874 as detailed in https://github.com/pulumi/pulumi-gcp/issues/1874#issuecomment-2030188438 so I am going to let it be for now.

EDIT: Discovered this was false. PlanResourceChange change indeed fixes https://github.com/pulumi/pulumi-gcp/issues/1874, so I'm picking this back up.

pulumi / pulumi-terraform-bridge