search

pulumi / pulumi-aws-iam

A Pulumi Multi Language Component for working with AWS IAM resources.
Apache License 2.0
6 stars 5 forks source link

panic: reflect.Set: value of type pulumi.AnyOutput is not assignable to type pulumi.String #1

Closed quodlibetor closed 2 years ago

quodlibetor commented 2 years ago

What happened?

I ran pulumi preview and got the following panic:

Diagnostics:
  pulumi:pulumi:Stack (production):
    panic: reflect.Set: value of type pulumi.AnyOutput is not assignable to type pulumi.String
    goroutine 56 [running]:
    reflect.Value.assignTo({0x1a401a0?, 0xc00014c3f0?, 0x194?}, {0x1aba9da, 0xb}, 0x199be40, 0x0)
        /opt/hostedtoolcache/go/1.18.2/x64/src/reflect/value.go:3064 +0x2ac
    reflect.Value.Set({0x199be40?, 0xc0004047d0?, 0x3?}, {0x1a401a0?, 0xc00014c3f0?, 0x199be40?})
        /opt/hostedtoolcache/go/1.18.2/x64/src/reflect/value.go:2090 +0xeb
    github.com/pulumi/pulumi/sdk/v3/go/pulumi.copyInputTo(0x18cc300?, {{0x19ecdc0?, 0xc0004809f0?}}, {0x199be40?, 0xc0004047d0?, 0x4?})
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.33.2/go/pulumi/provider.go:317 +0x19bb
    github.com/pulumi/pulumi/sdk/v3/go/pulumi.copyToSlice(0x18cc300?, {{0x18cb880?, 0xc00000c228?}}, {0x1c1d4c0?, 0x18cc300?}, {0x18cc300?, 0xc000052350?, 0xc000198ea0?})
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.33.2/go/pulumi/provider.go:495 +0x20a
    github.com/pulumi/pulumi/sdk/v3/go/pulumi.copyInputTo(0x1a27f20?, {{0x18cb880?, 0xc00000c228?}}, {0x18cc300?, 0xc000052350?, 0x18cc300?})
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.33.2/go/pulumi/provider.go:435 +0xe5a
    github.com/pulumi/pulumi/sdk/v3/go/pulumi.copyToStruct(0x1a29480?, {{0x1a27f20?, 0xc000480810?}}, {0x1c1d4c0, 0x1a29480}, {0x1a29480?, 0xc0003f2168?, 0x0?})
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.33.2/go/pulumi/provider.go:555 +0x29c
    github.com/pulumi/pulumi/sdk/v3/go/pulumi.copyInputTo(0xc000170f40?, {{0x1a27f20?, 0xc000480810?}}, {0x1a29480?, 0xc0003f2168?, 0x1a29480?})
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.33.2/go/pulumi/provider.go:437 +0xdfa
    github.com/pulumi/pulumi/sdk/v3/go/pulumi.constructInputsCopyTo(0xc0001e8280, 0xb0?, {0x18ad660?, 0xc0003f2160?})
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.33.2/go/pulumi/provider.go:595 +0xa30
    github.com/pulumi/pulumi/sdk/v3/go/pulumi/provider.linkedConstructInputsCopyTo(0x8?, 0x40?, {0x18ad660?, 0xc0003f2160?})
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.33.2/go/pulumi/provider_linked.go:44 +0x25
    github.com/pulumi/pulumi/sdk/v3/go/pulumi/provider.ConstructInputs.CopyTo(...)
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.33.2/go/pulumi/provider/provider.go:59
    github.com/pulumi/pulumi-aws-iam/pkg/provider.createNewResourceConstructor[...].func1({0xc00044a060?, 0x13}, {0xc0001e8280, 0xc000480b10?}, {0x1c0a320?, 0xc0000a6370?})
        /home/runner/work/pulumi-aws-iam/pulumi-aws-iam/provider/pkg/provider/provider.go:46 +0x9a
    github.com/pulumi/pulumi-aws-iam/pkg/provider.construct(0xc00007e480?, {0xc00044a048, 0x15}, {0xc00044a060, 0x13}, {0xc0001e8280?, 0xc000480b10?}, {0x1c0a320, 0xc0000a6370})
        /home/runner/work/pulumi-aws-iam/pulumi-aws-iam/provider/pkg/provider/provider.go:67 +0xc3
    github.com/pulumi/pulumi/sdk/v3/go/pulumi/provider.Construct.func1(0xc0001e8280?, {0xc00044a048?, 0x49?}, {0xc00044a060?, 0xc00040ede0?}, 0xc000484e20?, {0x1c0a320?, 0xc0000a6370?})
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.33.2/go/pulumi/provider/provider.go:38 +0x44
    github.com/pulumi/pulumi/sdk/v3/go/pulumi.construct({0x1c10c40, 0xc00040f2f0}, 0xc00054c000, 0x198aa00?, 0xc000484e70)
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.33.2/go/pulumi/provider.go:120 +0x969
    github.com/pulumi/pulumi/sdk/v3/go/pulumi/provider.linkedConstruct({0x1c10c40?, 0xc00040f2f0?}, 0x1009ced?, 0xc000171850?, 0x100cf85?)
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.33.2/go/pulumi/provider_linked.go:34 +0x25
    github.com/pulumi/pulumi/sdk/v3/go/pulumi/provider.Construct({0x1c10c40, 0xc00040f2f0}, 0x100ade5?, 0x1a103e0?, 0x1b13c08)
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.33.2/go/pulumi/provider/provider.go:35 +0x85
    github.com/pulumi/pulumi/pkg/v3/resource/provider.(*componentProvider).Construct(0x1c10c40?, {0x1c10c40?, 0xc00040f2f0?}, 0x40?)
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/pkg/v3@v3.33.2/resource/provider/component_provider.go:114 +0xff
    github.com/pulumi/pulumi/sdk/v3/proto/go._ResourceProvider_Construct_Handler.func1({0x1c10c40, 0xc00040f2f0}, {0x1a82e20?, 0xc00054c000})
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.33.2/proto/go/provider.pb.go:2694 +0x78
    github.com/grpc-ecosystem/grpc-opentracing/go/otgrpc.OpenTracingServerInterceptor.func1({0x1c10c40, 0xc00040e480}, {0x1a82e20, 0xc00054c000}, 0xc00040a340, 0xc000418360)
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/grpc-opentracing@v0.0.0-20180507213350-8e809c8a8645/go/otgrpc/server.go:57 +0x3f9
    github.com/pulumi/pulumi/sdk/v3/proto/go._ResourceProvider_Construct_Handler({0x1a4c260?, 0xc000141ea0}, {0x1c10c40, 0xc00040e480}, 0xc0004003c0, 0xc000137e20)
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.33.2/proto/go/provider.pb.go:2696 +0x138
    google.golang.org/grpc.(*Server).processUnaryRPC(0xc0002cb6c0, {0x1c17fd8, 0xc0004241a0}, 0xc0004b6000, 0xc00038d770, 0x2313ea0, 0x0)
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.46.2/server.go:1283 +0xcfd
    google.golang.org/grpc.(*Server).handleStream(0xc0002cb6c0, {0x1c17fd8, 0xc0004241a0}, 0xc0004b6000, 0x0)
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.46.2/server.go:1620 +0xa1b
    google.golang.org/grpc.(*Server).serveStreams.func1.2()
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.46.2/server.go:922 +0x98
    created by google.golang.org/grpc.(*Server).serveStreams.func1
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.46.2/server.go:920 +0x28a

    error: Resource monitor has terminated, shutting down
    error: an unhandled error occurred: Program exited with non-zero exit code: 1

Not sure if you have access to the state, but the ID of this preview was e86cd7af-4eaf-48fd-8dfc-5c3c0c87231a

Steps to reproduce

I'm not exactly certain where this came from. I do have a:

secret = Output.all(...).apply(aws.iam.SecretVersion(..))
policy = secret.arn.apply(lambda arn: aws.iam.Policy(...))
role = iam.EKSRole(
            f"{STACK}-app-role",
            role=iam.RoleArgs(
                name=f"{STACK}-app-role",
                policy_arns=[policy.arn],
            ),
        )

~And it can be fixed with an apply(..):~ EDIT: nevermind adding this apply just moves the error to pulumi up

secret = Output.all(...).apply(aws.iam.SecretVersion(..))
policy = secret.arn.apply(lambda arn: aws.iam.Policy(...))
role = policy.arn.apply(lambda policy_arn: iam.EKSRole(
            f"{STACK}-app-role",
            role=iam.RoleArgs(
                name=f"{STACK}-app-role",
                policy_arns=[policy_arn],
            ),
        ))

Expected Behavior

No panics, ideally a clear error message.

Actual Behavior

Panic.

Versions used

CLI
Version      3.35.2
Go Version   go1.17.11
Go Compiler  gc

Plugins
NAME        VERSION
aws         5.9.2
aws-iam     0.0.3
awsx        1.0.0-beta.9
docker      3.2.0
eks         0.40.0
kubernetes  3.19.4
python      unknown

Host
OS       darwin
Version  12.4
Arch     x86_64

This project is written in python: executable='/Users/bwm/.asdf/shims/python3' version='3.10.3'

Backend
Name           pulumi.com
URL            https://app.pulumi.com/quodlibetor
User           quodlibetor
Organizations  quodlibetor

Dependencies:
NAME            VERSION
pip             22.0.4
pulumi-aws-iam  0.0.3
pulumi-awsx     1.0.0b9
pulumi-eks      0.40.0
setuptools      60.10.0
wheel           0.37.1

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

mikhailshilkov commented 2 years ago

Hi @quodlibetor sorry for the trouble. Could you please try to assemble a runnable repro for me to be able to get the same error? I may be wrong but the stack trace suggests you are using a Component (maybe, eks.Cluster?) but I don't see it in your snippet.

On a side note, you should try to refactor your code so that no resources are created inside apply callbacks. Arguments for SecretVersion and Policy should be able to accept outputs directly, without the need to run apply first. The problem with your code is that some apply may not run during preview due to unresolved values, and then you will not see your resources in preview (this is probably the cause of the error happening in preview vs. up).

quodlibetor commented 2 years ago

I'll try and get a minimal repro for this, it'll be a few days before I'm working on this project again, though. You're correct that this is in EKS.cluster.

Thanks for the tip, it makes sense. I'm using apply in resource creation in most cases because I'm doing a json.dumps of a few outputs. That serialized JSON then becomes inputs for further resources -- RDS endpoints for SecretVersion and generated ARNs for IAM policies. Are there other alternatives that I could use for serializing a value, or am I just doing something completely wrong?

mikhailshilkov commented 2 years ago

Thanks for the tip, it makes sense. I'm using apply in resource creation in most cases because I'm doing a json.dumps of a few outputs. That serialized JSON then becomes inputs for further resources -- RDS endpoints for SecretVersion and generated ARNs for IAM policies. Are there other alternatives that I could use for serializing a value, or am I just doing something completely wrong?

You should do the serialization inside an apply, that's fine. Assign the result to a new variable and then use that variable for the next resource args, outside the apply call. Like in this example: https://github.com/pulumi/examples/blob/df13f6d5259479c273a77802a295d2370e261494/azure-go-containerapps/main.go#L32-L57

quodlibetor commented 2 years ago

Assign the result to a new variable and then use that variable for the next resource args, outside the apply call.

Ah, duh, yeah that's obvious. Thank you.

I just created a new stack and can reproduce this error by just trying to create an iam.EKSRole with nothing else:

import pulumi_aws_iam as iam

iam.EKSRole(
    "app-role",
    role=iam.RoleArgs(
        name=f"app-eks-role",
        policy_arns=["foo"],
    ),
    cluster_service_accounts={
        "foo": [f"default:app"],
    },
)

gives me:

❯ pulumi version
v3.35.3

❯ pulumi preview
Previewing update (test)

View Live: https://app.pulumi.com/quodlibetor/whatchow/test/previews/9dc79b5e-f7de-43c2-8a15-db7b549574eb

     Type                 Name           Plan       Info
 +   pulumi:pulumi:Stack  whatchow-test  create     2 errors; 50 messages

Diagnostics:
  pulumi:pulumi:Stack (whatchow-test):
    panic: reflect.Set: value of type pulumi.AnyOutput is not assignable to type pulumi.String
    goroutine 14 [running]:
    reflect.Value.assignTo({0x1a401a0?, 0xc000434ee0?, 0x194?}, {0x1aba9da, 0xb}, 0x199be40, 0x0)
        /opt/hostedtoolcache/go/1.18.2/x64/src/reflect/value.go:3064 +0x2ac
    reflect.Value.Set({0x199be40?, 0xc000091080?, 0x0?}, {0x1a401a0?, 0xc000434ee0?, 0x199be40?})
        /opt/hostedtoolcache/go/1.18.2/x64/src/reflect/value.go:2090 +0xeb
    github.com/pulumi/pulumi/sdk/v3/go/pulumi.copyInputTo(0x18cc300?, {{0x18e59a0?, 0xc000090fd0?}}, {0x199be40?, 0xc000091080?, 0xc000592340?})
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.33.2/go/pulumi/provider.go:427 +0xc5b
    github.com/pulumi/pulumi/sdk/v3/go/pulumi.copyToSlice(0x18cc300?, {{0x18cb880?, 0xc00012f698?}}, {0x1c1d4c0?, 0x18cc300?}, {0x18cc300?, 0xc0001c81d0?, 0xc000141080?})
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.33.2/go/pulumi/provider.go:495 +0x20a
    github.com/pulumi/pulumi/sdk/v3/go/pulumi.copyInputTo(0x1a27f20?, {{0x18cb880?, 0xc00012f698?}}, {0x18cc300?, 0xc0001c81d0?, 0x18cc300?})
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.33.2/go/pulumi/provider.go:435 +0xe5a
    github.com/pulumi/pulumi/sdk/v3/go/pulumi.copyToStruct(0x1a29480?, {{0x1a27f20?, 0xc0003f9aa0?}}, {0x1c1d4c0, 0x1a29480}, {0x1a29480?, 0xc0000d9a28?, 0x0?})
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.33.2/go/pulumi/provider.go:555 +0x29c
    github.com/pulumi/pulumi/sdk/v3/go/pulumi.copyInputTo(0x1a27f20?, {{0x1a27f20?, 0xc0003f9aa0?}}, {0x1a29480?, 0xc0000d9a28?, 0x1a29480?})
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.33.2/go/pulumi/provider.go:437 +0xdfa
    github.com/pulumi/pulumi/sdk/v3/go/pulumi.constructInputsCopyTo(0xc0000c4640, 0xb0?, {0x18ad660?, 0xc0000d9a20?})
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.33.2/go/pulumi/provider.go:595 +0xa30
    github.com/pulumi/pulumi/sdk/v3/go/pulumi/provider.linkedConstructInputsCopyTo(0x8?, 0x40?, {0x18ad660?, 0xc0000d9a20?})
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.33.2/go/pulumi/provider_linked.go:44 +0x25
    github.com/pulumi/pulumi/sdk/v3/go/pulumi/provider.ConstructInputs.CopyTo(...)
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.33.2/go/pulumi/provider/provider.go:59
    github.com/pulumi/pulumi-aws-iam/pkg/provider.createNewResourceConstructor[...].func1({0xc00040a6c8?, 0x8}, {0xc0000c4640, 0xc0003f9b00?}, {0x1c0a320?, 0xc0000abd60?})
        /home/runner/work/pulumi-aws-iam/pulumi-aws-iam/provider/pkg/provider/provider.go:46 +0x9a
    github.com/pulumi/pulumi-aws-iam/pkg/provider.construct(0xc000580000?, {0xc0001500f0, 0x15}, {0xc00040a6c8, 0x8}, {0xc0000c4640?, 0xc0003f9b00?}, {0x1c0a320, 0xc0000abd60})
        /home/runner/work/pulumi-aws-iam/pulumi-aws-iam/provider/pkg/provider/provider.go:67 +0xc3
    github.com/pulumi/pulumi/sdk/v3/go/pulumi/provider.Construct.func1(0xc0000c4640?, {0xc0001500f0?, 0x3d?}, {0xc00040a6c8?, 0xc0003f8300?}, 0x0?, {0x1c0a320?, 0xc0000abd60?})
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.33.2/go/pulumi/provider/provider.go:38 +0x44
    github.com/pulumi/pulumi/sdk/v3/go/pulumi.construct({0x1c10c40, 0xc0003f87b0}, 0xc000258400, 0x198aa00?, 0xc000090580)
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.33.2/go/pulumi/provider.go:120 +0x969
    github.com/pulumi/pulumi/sdk/v3/go/pulumi/provider.linkedConstruct({0x1c10c40?, 0xc0003f87b0?}, 0x1009ced?, 0xc000593850?, 0x100cf85?)
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.33.2/go/pulumi/provider_linked.go:34 +0x25
    github.com/pulumi/pulumi/sdk/v3/go/pulumi/provider.Construct({0x1c10c40, 0xc0003f87b0}, 0x100ade5?, 0x1a103e0?, 0x1b13c08)
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.33.2/go/pulumi/provider/provider.go:35 +0x85
    github.com/pulumi/pulumi/pkg/v3/resource/provider.(*componentProvider).Construct(0x1c10c40?, {0x1c10c40?, 0xc0003f87b0?}, 0x40?)
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/pkg/v3@v3.33.2/resource/provider/component_provider.go:114 +0xff
    github.com/pulumi/pulumi/sdk/v3/proto/go._ResourceProvider_Construct_Handler.func1({0x1c10c40, 0xc0003f87b0}, {0x1a82e20?, 0xc000258400})
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.33.2/proto/go/provider.pb.go:2694 +0x78
    github.com/grpc-ecosystem/grpc-opentracing/go/otgrpc.OpenTracingServerInterceptor.func1({0x1c10c40, 0xc0000a7890}, {0x1a82e20, 0xc000258400}, 0xc000140a80, 0xc00012f3e0)
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/grpc-opentracing@v0.0.0-20180507213350-8e809c8a8645/go/otgrpc/server.go:57 +0x3f9
    github.com/pulumi/pulumi/sdk/v3/proto/go._ResourceProvider_Construct_Handler({0x1a4c260?, 0xc000115ef0}, {0x1c10c40, 0xc0000a7890}, 0xc0000a4720, 0xc000141e20)
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.33.2/proto/go/provider.pb.go:2696 +0x138
    google.golang.org/grpc.(*Server).processUnaryRPC(0xc0003536c0, {0x1c17fd8, 0xc0001bc340}, 0xc000448240, 0xc0003f9800, 0x2313ea0, 0x0)
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.46.2/server.go:1283 +0xcfd
    google.golang.org/grpc.(*Server).handleStream(0xc0003536c0, {0x1c17fd8, 0xc0001bc340}, 0xc000448240, 0x0)
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.46.2/server.go:1620 +0xa1b
    google.golang.org/grpc.(*Server).serveStreams.func1.2()
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.46.2/server.go:922 +0x98
    created by google.golang.org/grpc.(*Server).serveStreams.func1
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.46.2/server.go:920 +0x28a

    error: Resource monitor has terminated, shutting down
    error: an unhandled error occurred: Program exited with non-zero exit code: 1
mikhailshilkov commented 2 years ago

Thank you for the repro @quodlibetor! The code and stack trace point it towards pulumi-aws-iam so I moved the issue to that repo cc @zchase

zchase commented 2 years ago

@quodlibetor This is fixed in the latest version 0.0.5. Thanks for filing the issue!!

Also, it is important to note that your repo will still throw an error because the Cluster Names under clusterServiceAccounts need to exist and the policyARN in the Role needs to a be proper ARN. Feel free to re-open the issue if you are still hitting any problems!

quodlibetor commented 2 years ago

Great, thanks! Yeah the repro there is known-broken, I just thought the error should be human-parseable, not just a panic.