search

FalcoSuessgott / vault-kubernetes-kms

Encrypt Kubernetes Secrets using Hashicorp Vault as the KMS Provider
https://falcosuessgott.github.io/vault-kubernetes-kms/
MIT License
29 stars 1 forks source link

Signal SIGSEGV: segmentation violation #85

Closed stephan2012 closed 3 months ago

stephan2012 commented 3 months ago

The first attempt to actually connect the plugin and the API server resulted in a SIGSEGV:

Jul 15 18:44:23 n0251 systemd[1]: Started vault-kms-plugin.service - Hashicorp Vault KMS Plugin.
Jul 15 18:44:23 n0251 vault-kubernetes-kms[710649]: {"level":"info","timestamp":"2024-07-15T18:44:23.842+0200","caller":"cmd/plugin.go:97","message":"starting kms plugin","socket":"unix:///var/lib/vault-kms/vault-kms.sock","debug":true,"vault-address":"https://vault-test.<redacted>:8200","vault-namespace":"","vault-token":"hvs.<redacted>","vault-k8s-mount":"kubernetes","vault-k8s-role":"","vault-transit-mount":"lab2/transit","vault-transit-key":"kms"}
Jul 15 18:44:23 n0251 vault-kubernetes-kms[710649]: {"level":"info","timestamp":"2024-07-15T18:44:23.892+0200","caller":"cmd/plugin.go:130","message":"Successfully authenticated to vault"}
Jul 15 18:44:23 n0251 vault-kubernetes-kms[710649]: {"level":"info","timestamp":"2024-07-15T18:44:23.892+0200","caller":"cmd/plugin.go:137","message":"Successfully created unix socket","socket":"/var/lib/vault-kms/vault-kms.sock"}
Jul 15 18:44:23 n0251 vault-kubernetes-kms[710649]: {"level":"info","timestamp":"2024-07-15T18:44:23.892+0200","caller":"cmd/plugin.go:144","message":"Listening for connection"}
Jul 15 18:44:23 n0251 vault-kubernetes-kms[710649]: {"level":"info","timestamp":"2024-07-15T18:44:23.892+0200","caller":"cmd/plugin.go:154","message":"Successfully registered kms plugin v1"}
Jul 15 18:44:23 n0251 vault-kubernetes-kms[710649]: {"level":"info","timestamp":"2024-07-15T18:44:23.892+0200","caller":"cmd/plugin.go:159","message":"Successfully registered kms plugin v2"}
Jul 15 18:47:15 n0251 vault-kubernetes-kms[710649]: {"level":"info","timestamp":"2024-07-15T18:47:15.419+0200","caller":"vault/client.go:119","message":"successfully refreshed token"}
Jul 15 18:47:15 n0251 vault-kubernetes-kms[710649]: panic: runtime error: invalid memory address or nil pointer dereference
Jul 15 18:47:15 n0251 vault-kubernetes-kms[710649]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0x90d5a9]
Jul 15 18:47:15 n0251 vault-kubernetes-kms[710649]: goroutine 19 [running]:
Jul 15 18:47:15 n0251 vault-kubernetes-kms[710649]: github.com/FalcoSuessgott/vault-kubernetes-kms/pkg/vault.(*Client).GetKeyVersions(0xc000256d50)
Jul 15 18:47:15 n0251 vault-kubernetes-kms[710649]:         /home/runner/work/vault-kubernetes-kms/vault-kubernetes-kms/pkg/vault/transit.go:78 +0xc9
Jul 15 18:47:15 n0251 vault-kubernetes-kms[710649]: github.com/FalcoSuessgott/vault-kubernetes-kms/pkg/plugin.(*PluginV2).Status(0xc0001821e8, {0xa60140?, 0xc0000a0060?}, 0xc0000e5940?)
Jul 15 18:47:15 n0251 vault-kubernetes-kms[710649]:         /home/runner/work/vault-kubernetes-kms/vault-kubernetes-kms/pkg/plugin/plugin_v2.go:73 +0xb6
Jul 15 18:47:15 n0251 vault-kubernetes-kms[710649]: k8s.io/kms/apis/v2._KeyManagementService_Status_Handler.func1({0xb8e998, 0xc0000b62a0}, {0xa60140?, 0xc0000a0060})
Jul 15 18:47:15 n0251 vault-kubernetes-kms[710649]:         /home/runner/go/pkg/mod/k8s.io/kms@v0.29.4/apis/v2/api.pb.go:494 +0x72
Jul 15 18:47:15 n0251 vault-kubernetes-kms[710649]: github.com/FalcoSuessgott/vault-kubernetes-kms/pkg/grpc.UnaryServerInterceptor({0xb8e998?, 0xc0000b62a0?}, {0xa60140?, 0xc0000a0060?}, 0xc0000a9998?, 0x9f5c20?)
Jul 15 18:47:15 n0251 vault-kubernetes-kms[710649]:         /home/runner/work/vault-kubernetes-kms/vault-kubernetes-kms/pkg/grpc/grpc.go:13 +0x42
Jul 15 18:47:15 n0251 vault-kubernetes-kms[710649]: k8s.io/kms/apis/v2._KeyManagementService_Status_Handler({0xa2d100?, 0xc0001821e8}, {0xb8e998, 0xc0000b62a0}, 0xc0000de000, 0xaed0b0)
Jul 15 18:47:15 n0251 vault-kubernetes-kms[710649]:         /home/runner/go/pkg/mod/k8s.io/kms@v0.29.4/apis/v2/api.pb.go:496 +0x135
Jul 15 18:47:15 n0251 vault-kubernetes-kms[710649]: google.golang.org/grpc.(*Server).processUnaryRPC(0xc0001bc600, {0xb8e998, 0xc0000b6210}, {0xb91e70, 0xc00032c300}, 0xc0000d2000, 0xc00019d590, 0xf8afa0, 0x0)
Jul 15 18:47:15 n0251 vault-kubernetes-kms[710649]:         /home/runner/go/pkg/mod/google.golang.org/grpc@v1.65.0/server.go:1379 +0xe23
Jul 15 18:47:15 n0251 vault-kubernetes-kms[710649]: google.golang.org/grpc.(*Server).handleStream(0xc0001bc600, {0xb91e70, 0xc00032c300}, 0xc0000d2000)
Jul 15 18:47:15 n0251 vault-kubernetes-kms[710649]:         /home/runner/go/pkg/mod/google.golang.org/grpc@v1.65.0/server.go:1790 +0x1016
Jul 15 18:47:15 n0251 vault-kubernetes-kms[710649]: google.golang.org/grpc.(*Server).serveStreams.func2.1()
Jul 15 18:47:15 n0251 vault-kubernetes-kms[710649]:         /home/runner/go/pkg/mod/google.golang.org/grpc@v1.65.0/server.go:1029 +0x8b
Jul 15 18:47:15 n0251 vault-kubernetes-kms[710649]: created by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 44
Jul 15 18:47:15 n0251 vault-kubernetes-kms[710649]:         /home/runner/go/pkg/mod/google.golang.org/grpc@v1.65.0/server.go:1040 +0x135
Jul 15 18:47:15 n0251 systemd[1]: vault-kms-plugin.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Jul 15 18:47:15 n0251 systemd[1]: vault-kms-plugin.service: Failed with result 'exit-code'.

Here's my EncryptionConfiguration:

apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
- providers:
  - kms:
      apiVersion: v2
      endpoint: unix:///var/lib/vault-kms/vault-kms.sock
      name: vault-kubernetes-kms
  - identity: {}
  resources:
  - secrets

The panic could be related to the transit engine path. I'll check this, but the plugin should never die from a SIGSEGV.

Please let me know if I can create more debug info somehow.

stephan2012 commented 3 months ago

Okay, the key in the transit engine was missing. Nevertheless, a suitable error message would be much appreciated. :-)