Receiving a `panic: cannot deep copy resource.Computed` panic when referencing `gcp:serviceAccount:Key` privateKey in yaml

zbuchheit commented 6 months ago

What happened?

When attempting to use Pulumi Yaml and referencing the privateKey of gcp:serviceAccount:Key to use as a kubernetes secret, I hit a panic error.

Example

name: yaml-k8s
runtime: yaml
description: A minimal Kubernetes Pulumi YAML program
resources:
  serviceAccount:
    type: gcp:serviceaccount:Account
    properties:
      accountId: zbuchheit-temp-sa
      displayName: Zbuchheit Temp SA
  sa-key:
    type: gcp:serviceaccount:Key
    properties:
      serviceAccountId: ${serviceAccount.accountId}
  secret:
    type: kubernetes:core/v1:Secret
    properties:
      type: Opaque
      metadata:
        name: mysecret
        namespace: default
      data:
        password-filejson: ${sa-key.privateKey} #Causing Panic

Error

panic: cannot deep copy resource.Computed
    goroutine 84 [running]:
    k8s.io/apimachinery/pkg/runtime.DeepCopyJSONValue({0x1046c02a0?, 0x140012fc700?})
        /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.29.0/pkg/runtime/converter.go:639 +0x22c
    k8s.io/apimachinery/pkg/runtime.DeepCopyJSONValue({0x104618b80?, 0x140012eb9b0?})
        /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.29.0/pkg/runtime/converter.go:623 +0x260
    k8s.io/apimachinery/pkg/runtime.DeepCopyJSON(...)
        /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.29.0/pkg/runtime/converter.go:608
    k8s.io/apimachinery/pkg/apis/meta/v1/unstructured.NestedMap(0x14000765118?, {0x14000765148?, 0x140007651b8?, 0x1039caf08?})
        /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.29.0/pkg/apis/meta/v1/unstructured/helpers.go:189 +0x38
    github.com/pulumi/pulumi-kubernetes/provider/v4/pkg/clients.normalizeSecretData(0x1400070b440)
        /home/runner/work/pulumi-kubernetes/pulumi-kubernetes/provider/pkg/clients/unstructured.go:149 +0x50
    github.com/pulumi/pulumi-kubernetes/provider/v4/pkg/clients.normalizeSecret(0x1400070b440)
        /home/runner/work/pulumi-kubernetes/pulumi-kubernetes/provider/pkg/clients/unstructured.go:120 +0x204
    github.com/pulumi/pulumi-kubernetes/provider/v4/pkg/clients.Normalize(0x104891fe0?)
        /home/runner/work/pulumi-kubernetes/pulumi-kubernetes/provider/pkg/clients/unstructured.go:75 +0xd0
    github.com/pulumi/pulumi-kubernetes/provider/v4/pkg/provider.normalizeInputs(0x1400070b440)
        /home/runner/work/pulumi-kubernetes/pulumi-kubernetes/provider/pkg/provider/provider.go:2751 +0x88
    github.com/pulumi/pulumi-kubernetes/provider/v4/pkg/provider.(*kubeProvider).Check(0x14000422f00, {0x104a3e570, 0x140012eb800}, 0x140012c3500)
        /home/runner/work/pulumi-kubernetes/pulumi-kubernetes/provider/pkg/provider/provider.go:1339 +0x48c
    github.com/pulumi/pulumi/sdk/v3/proto/go._ResourceProvider_Check_Handler.func1({0x104a3e570, 0x140012eb800}, {0x104845fc0?, 0x140012c3500})
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.96.2/proto/go/provider_grpc.pb.go:557 +0x74
    github.com/grpc-ecosystem/grpc-opentracing/go/otgrpc.OpenTracingServerInterceptor.func1({0x104a3e570, 0x140012eb200}, {0x104845fc0, 0x140012c3500}, 0x140008b0740, 0x140012b30c8)
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/grpc-opentracing@v0.0.0-20180507213350-8e809c8a8645/go/otgrpc/server.go:57 +0x2e8
    github.com/pulumi/pulumi/sdk/v3/proto/go._ResourceProvider_Check_Handler({0x10499fb80?, 0x14000422f00}, {0x104a3e570, 0x140012eb200}, 0x140012d6e70, 0x14000122940)
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.96.2/proto/go/provider_grpc.pb.go:559 +0x12c
    google.golang.org/grpc.(*Server).processUnaryRPC(0x140008e45a0, {0x104a4cb20, 0x14000622680}, 0x140012e4480, 0x1400061e5d0, 0x106261a50, 0x0)
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.58.3/server.go:1374 +0xb44
    google.golang.org/grpc.(*Server).handleStream(0x140008e45a0, {0x104a4cb20, 0x14000622680}, 0x140012e4480, 0x0)
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.58.3/server.go:1751 +0x7cc
    google.golang.org/grpc.(*Server).serveStreams.func1.1()
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.58.3/server.go:986 +0xb4
    created by google.golang.org/grpc.(*Server).serveStreams.func1 in goroutine 27
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.58.3/server.go:997 +0x160

Output of `pulumi about`

CLI          
Version      3.100.0
Go Version   go1.21.5
Go Compiler  gc

# Manually filling in plugins
Plugins
NAME        VERSION
gcp         6.67.1 # Error present in latest version as well
kubernetes 4.6.1
yaml        1.4.5

Host     
OS       darwin
Version  14.2.1
Arch     arm64

This project is written in yaml
Current Stack: zbuchheit-pulumi-corp/yaml-k8s/dev

Backend        
Name           pulumi.com
URL            https://app.pulumi.com/zbuchheit-pulumi-corp
User           zbuchheit-pulumi-corp
Organizations  zbuchheit-pulumi-corp,
Token type     personal

Additional context

It seem to be present in the create and preview logic. If I create the Key resource first without referencing the privateKey, it will succeed going forward. The same code when converted to typescript works without issue.

Also worth noting the output privateKey is a base64 encoded value.

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

justinvp commented 5 months ago

https://github.com/pulumi/pulumi-kubernetes/pull/2846 addresses the panic in the Kubernetes provider. There's a question of whether the way YAML sends data impacts any other provider and whether we need to change YAML to send data in the same way as other SDKs.

justinvp commented 4 months ago

Closing as fixed by https://github.com/pulumi/pulumi-kubernetes/pull/2846. It's unclear if we need to make any changes in YAML itself.

pulumi / pulumi-yaml