search

pulumi / pulumi-aws

An Amazon Web Services (AWS) Pulumi resource package, providing multi-language access to AWS
Apache License 2.0
459 stars 155 forks source link

Error on refresh after import for pf based resources with tags #4204

Closed corymhall closed 2 months ago

corymhall commented 3 months ago

Describe what happened

Steps to reproduce:

  1. Using the below program run pulumi up
  2. Delete the appconfig:Environment resource pulumi state delete 'urn:pulumi:dev::test-aws-pf::aws:appconfig/environment:Environment::res'
  3. Import the resource using the id in the outputs pulumi import aws:appconfig/environment:Environment res "8i7rjaa:hi75mq1" --provider aws-provider=urn:pulumi:dev::test-aws-pf::pulumi:providers:aws::aws-provider
  4. Look at the state for the resource and notice that the inputs.tags have a "__defaults": [] property. pulumi stack export 
    {
"deployment": {
    "resources": [
        {
            "urn": "urn:pulumi:dev::test-aws-pf::aws:appconfig/environment:Environment::res",
            "custom": true,
            "id": "86413we:7szszb8",
            "type": "aws:appconfig/environment:Environment",
            "inputs": {
                "tags": {
                    "LocalTag": "foo",
                    "__defaults": []
                },
                "tagsAll": {
                    "GlobalTag": "bar",
                    "LocalTag": "foo",
                    "__defaults": []
                }
            },
            "outputs": {
                "tags": {
                    "LocalTag": "foo"
                },
                "tagsAll": {
                    "GlobalTag": "bar",
                    "LocalTag": "foo"
                }
            }
        }
    ]
}
}
  5. Try and run a pulumi refresh and get an error 
    aws:appconfig:Environment (res):
error: Preview failed: objectEncoder failed on property "tags_all": encMap failed on {[]}
  6. Run a pulumi up and view the state. Notice that the "__defaults": [] property is no longer there 
    {
"deployment": {
    "resources": [
        {
            "urn": "urn:pulumi:dev::test-aws-pf::aws:appconfig/environment:Environment::res",
            "custom": true,
            "id": "86413we:7szszb8",
            "type": "aws:appconfig/environment:Environment",
            "inputs": {
                "tags": {
                    "LocalTag": "foo"
                },
                "tagsAll": {
                    "GlobalTag": "bar",
                    "LocalTag": "foo"
                }
            },
            "outputs": {
                "tags": {
                    "LocalTag": "foo"
                },
                "tagsAll": {
                    "GlobalTag": "bar",
                    "LocalTag": "foo"
                }
            }
        }
    ]
}
}
  7. Run a pulumi refresh and it will succeed.

Sample program

name: test-aws-pf
runtime: yaml
resources:
  aws-provider:
    type: pulumi:providers:aws
    properties:
      defaultTags:
        tags:
          GlobalTag: bar
  app:
    type: aws:appconfig:Application
    properties:
      tags:
        LocalTag: foo
    options:
      provider: ${aws-provider}
  res:
    type: aws:appconfig:Environment
    options:
      provider: ${aws-provider}
    properties:
      applicationId: ${app.id}
      name: tags-pf-GwmaM
      tags:
        LocalTag: foo
outputs:
  actual: ${res.tags}
  urn: ${res.urn}
  id: ${res.id}
  providerUrn: ${aws-provider.urn}

Log output

Here are the grpc logs

CLI import

{
  "method": "/pulumirpc.ResourceProvider/Read",
  "request": {
    "id": "86413we:7szszb8",
    "urn": "urn:pulumi:dev::test-aws-1655-pf::aws:appconfig/environment:Environment::res",
    "properties": {}
  },
  "response": {
    "id": "86413we:7szszb8",
    "properties": {
      "applicationId": "7szszb8",
      "arn": "arn:aws:appconfig:us-east-2:616138583583:application/7szszb8/environment/86413we",
      "description": "",
      "environmentId": "86413we",
      "id": "86413we:7szszb8",
      "monitors": [],
      "name": "tags-pf-GwmaM",
      "state": "ReadyForDeployment",
      "tags": {
        "LocalTag": "foo"
      },
      "tagsAll": {
        "GlobalTag": "bar",
        "LocalTag": "foo"
      }
    },
    "inputs": {
      "applicationId": "7szszb8",
      "name": "tags-pf-GwmaM",
      "tags": {
        "LocalTag": "foo",
        "__defaults": []
      },
      "tagsAll": {
        "GlobalTag": "bar",
        "LocalTag": "foo",
        "__defaults": []
      }
    }
  },
  "metadata": {
    "kind": "resource",
    "mode": "client",
    "name": "aws"
  }
}
{
  "method": "/pulumirpc.ResourceProvider/Check",
  "request": {
    "urn": "urn:pulumi:dev::test-aws-1655-pf::aws:appconfig/environment:Environment::res",
    "olds": {
      "applicationId": "7szszb8",
      "name": "tags-pf-GwmaM",
      "tags": {
        "LocalTag": "foo",
        "__defaults": []
      },
      "tagsAll": {
        "GlobalTag": "bar",
        "LocalTag": "foo",
        "__defaults": []
      }
    },
    "news": {
      "applicationId": "7szszb8",
      "name": "tags-pf-GwmaM",
      "tags": {
        "LocalTag": "foo",
        "__defaults": []
      },
      "tagsAll": {
        "GlobalTag": "bar",
        "LocalTag": "foo",
        "__defaults": []
      }
    },
    "randomSeed": "td8uAv54LOcLZv1pOpqtyQIvFjgk6mWP2jp8JvoEHl8="
  },
  "response": {
    "inputs": {
      "applicationId": "7szszb8",
      "name": "tags-pf-GwmaM",
      "tags": {
        "GlobalTag": "bar",
        "LocalTag": "foo",
        "__defaults": ""
      },
      "tagsAll": {
        "GlobalTag": "bar",
        "LocalTag": "foo",
        "__defaults": ""
      }
    }
  },
  "metadata": {
    "kind": "resource",
    "mode": "client",
    "name": "aws"
  }
}

Refresh

{
  "method": "/pulumirpc.ResourceProvider/Read",
  "request": {
    "id": "86413we:7szszb8",
    "urn": "urn:pulumi:dev::test-aws-1655-pf::aws:appconfig/environment:Environment::res",
    "properties": {
      "applicationId": "7szszb8",
      "arn": "arn:aws:appconfig:us-east-2:616138583583:application/7szszb8/environment/86413we",
      "description": "",
      "environmentId": "86413we",
      "id": "86413we:7szszb8",
      "monitors": [],
      "name": "tags-pf-GwmaM",
      "state": "ReadyForDeployment",
      "tags": {
        "LocalTag": "foo"
      },
      "tagsAll": {
        "GlobalTag": "bar",
        "LocalTag": "foo"
      }
    },
    "inputs": {
      "applicationId": "7szszb8",
      "name": "tags-pf-GwmaM",
      "tags": {
        "LocalTag": "foo",
        "__defaults": []
      },
      "tagsAll": {
        "GlobalTag": "bar",
        "LocalTag": "foo",
        "__defaults": []
      }
    }
  },
  "response": {
    "id": "86413we:7szszb8",
    "properties": {
      "applicationId": "7szszb8",
      "arn": "arn:aws:appconfig:us-east-2:616138583583:application/7szszb8/environment/86413we",
      "description": "",
      "environmentId": "86413we",
      "id": "86413we:7szszb8",
      "monitors": [],
      "name": "tags-pf-GwmaM",
      "state": "ReadyForDeployment",
      "tags": {
        "LocalTag": "foo"
      },
      "tagsAll": {
        "GlobalTag": "bar",
        "LocalTag": "foo"
      }
    },
    "inputs": {
      "applicationId": "7szszb8",
      "name": "tags-pf-GwmaM",
      "tags": {
        "LocalTag": "foo"
      },
      "tagsAll": {
        "GlobalTag": "bar",
        "LocalTag": "foo"
      }
    }
  },
  "metadata": {
    "kind": "resource",
    "mode": "client",
    "name": "aws"
  }
}
{
  "method": "/pulumirpc.ResourceProvider/Diff",
  "request": {
    "id": "86413we:7szszb8",
    "urn": "urn:pulumi:dev::test-aws-1655-pf::aws:appconfig/environment:Environment::res",
    "olds": {
      "applicationId": "7szszb8",
      "arn": "arn:aws:appconfig:us-east-2:616138583583:application/7szszb8/environment/86413we",
      "description": "",
      "environmentId": "86413we",
      "id": "86413we:7szszb8",
      "monitors": [],
      "name": "tags-pf-GwmaM",
      "state": "ReadyForDeployment",
      "tags": {
        "LocalTag": "foo"
      },
      "tagsAll": {
        "GlobalTag": "bar",
        "LocalTag": "foo"
      }
    },
    "news": {
      "applicationId": "7szszb8",
      "name": "tags-pf-GwmaM",
      "tags": {
        "LocalTag": "foo",
        "__defaults": []
      },
      "tagsAll": {
        "GlobalTag": "bar",
        "LocalTag": "foo",
        "__defaults": []
      }
    },
    "oldInputs": {
      "applicationId": "7szszb8",
      "name": "tags-pf-GwmaM",
      "tags": {
        "LocalTag": "foo"
      },
      "tagsAll": {
        "GlobalTag": "bar",
        "LocalTag": "foo"
      }
    }
  },
  "errors": [
    "rpc error: code = Unknown desc = objectEncoder failed on property \"tags_all\": encMap failed on {[]}"
  ],
  "metadata": {
    "kind": "resource",
    "mode": "client",
    "name": "aws"
  }
}

Affected Resource(s)

Any pf based resource.

Output of pulumi about

CLI          
Version      3.121.0
Go Version   go1.22.4
Go Compiler  gc

Plugins
KIND      NAME  VERSION
resource  aws   unknown
language  yaml  unknown

Host     
OS       darwin
Version  14.5
Arch     arm64

This project is written in yaml

Current Stack: corymhall/test-aws-1655-pf/dev

TYPE                                   URN
pulumi:pulumi:Stack                    urn:pulumi:dev::test-aws-1655-pf::pulumi:pulumi:Stack::test-aws-1655-pf-dev
pulumi:providers:aws                   urn:pulumi:dev::test-aws-1655-pf::pulumi:providers:aws::aws-provider
aws:appconfig/application:Application  urn:pulumi:dev::test-aws-1655-pf::aws:appconfig/application:Application::app

Found no pending operations associated with dev

Backend        
Name           pulumi.com
URL            https://app.pulumi.com/corymhall
User           corymhall
Organizations  corymhall
Token type     personal

No dependencies found

Pulumi locates its logs in /var/folders/3b/6mr1jkqx7r797ff75k27jfjc0000gn/T/ by default

Additional context

The same thing can be done with sdkv2 based resources without any error. sdkv2 based resource still have the "__defaults": [] key written to the state, but it does not cause an error to be thrown on refresh.

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

corymhall commented 2 months ago

@t0yv0

t0yv0 commented 2 months ago

I'm pretty sure this is coming from https://github.com/pulumi/pulumi-terraform-bridge/blob/master/pkg/tfbridge/schema.go#L1743 or around that code, where PF bridge reuses ExtractInputsFromOutputs function from the old bridge but PF bridge does not support or expect __defaults. I'll file a matching issue in the bridge as this is the right place to resolve it.