Pulumi tries to delete `gcp:cloudrun:IamPolicy` even if it is already deleted

[remorses]

What happened?

Pulumi tries to delete gcp:cloudrun:IamPolicy even if it is already deleted

I manaully deleted a cloud run service, pulumi detects that the service is deleted but tries to delte the iam policy (which no longer exists)

Steps to reproduce

• Deploy the following stack
• delete the service manually in console
• try to re deploy

const service = new gcp.cloudrun.Service(NAME, {
        location,
        project,
        // metadata: {
        //     namespace: gcp.config.project,
        //     annotations: {
        //         'run.googleapis.com/ingress': 'internal',
        //         'run.googleapis.com/ingress-status': 'internal',
        //     },
        // },
        template: {
            spec: {
                containers: [
                    {
                        image: "gcr.io/cloudrun/hello",
                        resources: {
                            limits: {
                                memory: '256M',
                            },
                        },
                    },
                ],
                containerConcurrency: 20,

            },
        },
        traffics: [
            {
                percent: 100,
                latestRevision: true,
            },
        ],
    })
    const noauthIAMPolicy = gcp.organizations.getIAMPolicy({
        bindings: [
            {
                role: 'roles/run.invoker',
                members: ['allUsers'],
            },
        ],
    })
    const noauthIamPolicy = new gcp.cloudrun.IamPolicy(
        'noauthIamPolicy',
        {
            location: service.location,
            project: service.project,
            service: service.name,
            policyData: noauthIAMPolicy.then(
                (noauthIAMPolicy) => noauthIAMPolicy.policyData,
            ),
        },
    )

Expected Behavior

it should not try to delete a deleted resource

Actual Behavior

fails with

error: deleting urn:pulumi:production::salespack-gcp-cloudrun::gcp:cloudrun/iamPolicy:IamPolicy::noauthIamPolicy: 1 error occurred:
        * Error setting IAM policy for cloudrun service "xxx": googleapi: Error 404: Resource 'xxxx' of kind 'SERVICE' in region 'us-east1' in project 'xxx' does not exist.

Output of pulumi about

Version      3.43.1
Go Version   go1.19.2
Go Compiler  gc

Plugins
NAME    VERSION
nodejs  unknown

Host     
OS       darwin
Version  12.6
Arch     x86_64

This project is written in nodejs: executable='/Users/morse/.local/share/pnpm/node' version='v16.16.0'

Current Stack: production

TYPE                              URN
pulumi:pulumi:Stack               urn:pulumi:production::salespack-gcp-cloudrun::pulumi:pulumi:Stack::salespack-gcp-cloudrun-production
pulumi:providers:gcp              urn:pulumi:production::salespack-gcp-cloudrun::pulumi:providers:gcp::default_6_40_0
gcp:cloudtasks/queue:Queue        urn:pulumi:production::salespack-gcp-cloudrun::gcp:cloudtasks/queue:Queue::salespack-emails
gcp:cloudrun/service:Service      urn:pulumi:production::salespack-gcp-cloudrun::gcp:cloudrun/service:Service::warmup
gcp:cloudrun/iamPolicy:IamPolicy  urn:pulumi:production::salespack-gcp-cloudrun::gcp:cloudrun/iamPolicy:IamPolicy::noauthIamPolicy
pulumi:providers:gcp              urn:pulumi:production::salespack-gcp-cloudrun::pulumi:providers:gcp::default_6_39_0
gcp:cloudrun/iamPolicy:IamPolicy  urn:pulumi:production::salespack-gcp-cloudrun::gcp:cloudrun/iamPolicy:IamPolicy::noauthIamPolicy

Found no pending operations associated with production

Backend        
Name           pulumi.com
URL            https://app.pulumi.com/remorses
User           remorses
Organizations  remorses, EpicGames

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[remorses]

i managed to fix my state with pulumi state delete urn:pulumi:production::xxx-gcp-cloudrun::gcp:cloudrun/iamPolicy:IamPolicy::noauthIamPolicy

[guineveresaenger]

Hi @remorses - sometimes when a Pulumi stack gets out of sync with its backend, this can happen - in case you weren't aware, we have Pulumi Refresh for that situation!

Thank you for sharing your workaround as well. 🙇‍♀️

[remorses]

@guineveresaenger pulumi refresh did not work

Pulumi was detecting that the service was deleted but not the iam policy

[guineveresaenger]

ah! that makes sense - so I have one more suggestion for the future for you:

If resources are dependent on one another, you can denote that relationship with a parent resource option.

This wouldn't've helped in this case since relationships must be established before reconciliation attempts with your stack, but it may be something for you to look into.