pulumi / pulumi-kubernetes

A Pulumi resource provider for Kubernetes to manage API resources and workloads in running clusters
https://www.pulumi.com/docs/reference/clouds/kubernetes/
Apache License 2.0
404 stars 115 forks source link

Ctrl-c leads to panic #3209

Closed lukehoban closed 1 day ago

lukehoban commented 6 days ago

Twice when I ctrl-c a deployment that is in process of deleting a Service I have seen this:

View in Browser (Ctrl+O): https://app.pulumi.com/upstarts/infra/dev/updates/16

     Type                                 Name       Status                  Info
     pulumi:pulumi:Stack                  infra-dev  **failed**              1 error; 21 messages
     └─ k8sx:component:ServiceDeployment  app                                
 --     └─ kubernetes:core/v1:Service     app        **deleting failed**     1 error

Diagnostics:
  pulumi:pulumi:Stack (infra-dev):
    panic: runtime error: invalid memory address or nil pointer dereference
    [signal SIGSEGV: segmentation violation code=0x2 addr=0x0 pc=0x10393c068]
    goroutine 15 [running]:
    github.com/pulumi/pulumi-kubernetes/provider/v4/pkg/provider.checkpointObject(0x140015152c0, 0x0, 0x1400131f5c0, {0x140012faea8, 0x2}, {0x14001300980, 0x1a})
        /home/runner/work/pulumi-kubernetes/pulumi-kubernetes/provider/pkg/provider/provider.go:2932 +0x38
    github.com/pulumi/pulumi-kubernetes/provider/v4/pkg/provider.(*kubeProvider).Delete(0x1400033c000, {0x1052226d0, 0x1400131f590}, 0x1400132e000)
        /home/runner/work/pulumi-kubernetes/pulumi-kubernetes/provider/pkg/provider/provider.go:2607 +0xb54
    github.com/pulumi/pulumi/sdk/v3/proto/go._ResourceProvider_Delete_Handler.func1({0x1052226d0?, 0x1400131f590?}, {0x104b0c940?, 0x1400132e000?})
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.127.0/proto/go/provider_grpc.pb.go:705 +0xd0
    github.com/grpc-ecosystem/grpc-opentracing/go/otgrpc.OpenTracingServerInterceptor.func1({0x1052226d0, 0x1400131e480}, {0x104b0c940, 0x1400132e000}, 0x140010a1e00, 0x140012dedb0)
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/grpc-opentracing@v0.0.0-20180507213350-8e809c8a8645/go/otgrpc/server.go:57 +0x2e4
    github.com/pulumi/pulumi/sdk/v3/proto/go._ResourceProvider_Delete_Handler({0x105130b60, 0x1400033c000}, {0x1052226d0, 0x1400131e480}, 0x140012ecb00, 0x14000a48440)
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.127.0/proto/go/provider_grpc.pb.go:707 +0x148
    google.golang.org/grpc.(*Server).processUnaryRPC(0x14000219000, {0x1052226d0, 0x1400131e3c0}, {0x10527aa20, 0x14000a5c180}, 0x14001307c20, 0x14000a4bbf0, 0x108d8c8c0, 0x0)
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.65.0/server.go:1379 +0xb58
    google.golang.org/grpc.(*Server).handleStream(0x14000219000, {0x10527aa20, 0x14000a5c180}, 0x14001307c20)
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.65.0/server.go:1790 +0xb20
    google.golang.org/grpc.(*Server).serveStreams.func2.1()
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.65.0/server.go:1029 +0x8c
    created by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 69
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.65.0/server.go:1040 +0x13c

    error: update failed

  kubernetes:core/v1:Service (app):
    error: error reading from server: EOF

Nearly certain this is a regression, we have typically handled ctrl-c gracefully in this provider.

blampe commented 2 days ago

@lukehoban I think this is a dupe of https://github.com/pulumi/pulumi-kubernetes/issues/3157, do you still see the issue on v4.18?

lukehoban commented 1 day ago

Ahh - I'm sure that was it - I was on 4.17.1. I'll close out for now - and reopen if I hit this again.