Provider returns wrong field

[pawelprazak]

What happened?

https://github.com/pulumi/pulumi-java/pull/597

The field deployment contains wrong URN: urn:pulumi:dev::iac-workshop-lambda-api-gateway::aws-apigateway:index:RestAPI::helloWorldApi

instead of: urn:pulumi:dev::iac-workshop-lambda-api-gateway::aws-apigateway:index:RestAPI$aws:apigateway/deployment:Deployment

according to schema

Java implementation is unable to deserialize this:

debug: Read response for resource: t=aws-apigateway:index:RestAPI, name=helloWorldApi, urn=urn:pulumi:dev::iac-workshop-lambda-api-gateway::aws-apigateway:index:RestAPI::helloWorldApi, id=, remote=true, data=fields {
      key: "api"
      value {
        struct_value {
          fields {
            key: "4dabf18193072939515e22adb298388d"
            value {
              string_value: "5cf8f73096256a8f31e491e813e4eb8e"
            }
          }
          fields {
            key: "id"
            value {
              string_value: ""
            }
          }
          fields {
            key: "urn"
            value {
              string_value: "urn:pulumi:dev::iac-workshop-lambda-api-gateway::aws-apigateway:index:RestAPI$aws:apigateway/restApi:RestApi::helloWorldApi"
            }
          }
        }
      }
    }
    fields {
      key: "deployment"
      value {
        struct_value {
          fields {
            key: "4dabf18193072939515e22adb298388d"
            value {
              string_value: "5cf8f73096256a8f31e491e813e4eb8e"
            }
          }
          fields {
            key: "urn"
            value {
              string_value: "urn:pulumi:dev::iac-workshop-lambda-api-gateway::aws-apigateway:index:RestAPI::helloWorldApi"
            }
          }
        }
      }
    }
    fields {
      key: "stage"
      value {
        struct_value {
          fields {
            key: "4dabf18193072939515e22adb298388d"
            value {
              string_value: "5cf8f73096256a8f31e491e813e4eb8e"
            }
          }
          fields {
            key: "id"
            value {
              string_value: ""
            }
          }
          fields {
            key: "urn"
            value {
              string_value: "urn:pulumi:dev::iac-workshop-lambda-api-gateway::aws-apigateway:index:RestAPI$aws:apigateway/stage:Stage::helloWorldApi"
            }
          }
        }
      }
    }

Steps to reproduce

https://github.com/pulumi/pulumi-java/pull/597

Expected Behavior

Getting a correct URN.

Actual Behavior

Getting a wrong URN.

Versions used

No response

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[t0yv0]

Could you please post the shortest Pulumi program that reproduces this. The link in "Steps to reproduce" is not actionable.

The next step would be to try to rewrite this program to TypeScript and trace what happens then.

I suspect, but would need to confirm, that the Java SDK is not handling resource references correctly in the serde.

This is a type reference in schema expecting that the type of "deployment" property is "Deployment":

     "$ref": "/aws/v5.4.0/schema.json#/resources/aws:apigateway%2Fdeployment:Deployment"

This is a reference to an object (resource) named helloWorldApi in a current stack. From https://pulumi-developer-docs.readthedocs.io/en/latest/providers/implementers-guide.html?highlight=URN#urns it would seem that the type inlined into the URN is unexpecetd (RestAPI instead of Deployment) but perhaps that is OK? SDK should not be validating that perhaps, but treating URNs as opaque. If there is an URN corresponding to this resource in the stack perhaps use it?

urn:pulumi:dev::iac-workshop-lambda-api-gateway::aws-apigateway:index:RestAPI::helloWorldApi

[t0yv0]

I'm guessing somewhat though, first need a repro I can trace to find out more.

[t0yv0]

OK the repro is in https://github.com/pulumi/pulumi-java/issues/594 embedded in the issue. Let me try that.

[t0yv0]

I could not reproduce in TypeScript because the provider does not install (https://github.com/pulumi/pulumi-aws-apigateway/issues/42) but I did reproduce in C#:

using Pulumi;
using Pulumi.Aws.S3;
using Pulumi.Aws.Iam;
using Pulumi.Aws.Lambda;
using Pulumi.AwsApiGateway;
using Pulumi.AwsApiGateway.Inputs;
using System.Collections.Generic;

class MyStack : Stack
{
    public MyStack()
    {

        var role = new Role("lambdaRole", new RoleArgs()
        {
            AssumeRolePolicy = @"

                            {
                              ""Version"": ""2012-10-17"",
                              ""Statement"": [
                                {
                                  ""Effect"": ""Allow"",
                                  ""Principal"": {
                                    ""Service"": ""lambda.amazonaws.com""
                                  },
                                  ""Action"": ""sts:AssumeRole""
                                }
                              ]
                            }
"
        });

        var rpaArgs = new RolePolicyAttachmentArgs()
        {
            Role = role.Name,
            PolicyArn = ManagedPolicy.AWSLambdaBasicExecutionRole.ToString()
        };

        var lambdaBasicExecutionRole = new RolePolicyAttachment(
            "basicExecutionRole", rpaArgs);

        var lambdaFunc = new Function("helloWorldFunction", new FunctionArgs()
        {
            Role = role.Arn,
            Handler = "index.handler",
            Runtime = Runtime.NodeJS12dX,
                Code = new AssetArchive(new Dictionary<string,AssetOrArchive>())
        });

        var restApi = new RestAPI("helloWorldApi", new RestAPIArgs()
        {
            Routes = {
                new RouteArgs()
                {
                    Path = "/",
                    Method = Method.GET,
                    EventHandler = lambdaFunc
                }
            }
        });

        // Create an AWS resource (S3 Bucket)
        var bucket = new Bucket("my-bucket");

        // Export the name of the bucket
        this.BucketName = bucket.Id;
    }

    [Output]
    public Output<string> BucketName { get; set; }
}

Refs:

  <ItemGroup>
    <PackageReference Include="Pulumi" Version="3.*" />
    <PackageReference Include="Pulumi.Aws" Version="5.*" />
    <PackageReference Include="Pulumi.AwsApiGateway" Version="0.0.8" />
  </ItemGroup>

Result:

    error: Running program '/Users/anton/tmp/repro/pulumi-java-594/cs-version/bin/Debug/netcoreapp3.1/test-csharp.dll' failed with an unhandled exception:
    System.InvalidCastException: Unable to cast object of type 'Pulumi.AwsApiGateway.RestAPI' to type 'Pulumi.Aws.ApiGateway.Deployment'.
       at void Pulumi.Serialization.OutputCompletionSource<T>.SetValue(OutputData<object> data)
       at async Task Pulumi.Deployment.CompleteResourceAsync(Resource resource, bool remote, Func<string, Resource> newDependency, ResourceArgs args, ResourceOptions options, ImmutableDictionary<string, IOutputCompletionSource> completionSources)

So it's encouraging we're having this issue in both C# and Java.

[t0yv0]

Okay. I did not get a chance to test Go yet but it sounds like all of Go/C#/Java have the same idea, that is when parsing a resource reference that's just an URN from the wire, they go ahead and try to parse type out of the URN and construct a value of that type.

So then the issue is indeed somewhere in how this URN got built:

Actual: urn:pulumi:dev::iac-workshop-lambda-api-gateway::aws-apigateway:index:RestAPI::helloWorldApi
Expect: urn:pulumi:dev::iac-workshop-lambda-api-gateway::aws-apigateway:index:Deployment::helloWorldApi

[pawelprazak]

Looking at other fields, by analogy, this is what the URN should look like: urn:pulumi:dev::iac-workshop-lambda-api-gateway::aws-apigateway:index:RestAPI$aws:apigateway/deployment:Deployment

[pawelprazak]

A kindly reminder that this issue was surfaced while porting the AWS Workshop and is required for that effort to continue. Thank you.

[pawelprazak]

If it helps, this is the exact place where the log "debug: Read response for resource: ..." is from: https://github.com/pulumi/pulumi-java/blob/cb3c1b15b423e8f20859b9cd5ea88cb47aba7cc9/sdk/java/pulumi/src/main/java/com/pulumi/deployment/internal/DeploymentImpl.java#L1147