uhthomas
commented
1 year ago Looks like this is quite a delicate operation too as the operator will crash if the secret for an lb is deleted.
{"level":"error","ts":1680727688.1724286,"logger":"service-reconciler","msg":"Tailscale proxy secret doesn't exist, but the corresponding StatefulSet tailscale/ts-jellyseerr-mvt4n already does. Something is wrong, please delete the StatefulSet.","service-ns":"media","service-name":"jellyseerr","stacktrace":"main.(*ServiceReconciler).createOrGetSecret\n\ttailscale.com/cmd/k8s-operator/operator.go:544\nmain.(*ServiceReconciler).maybeProvision\n\ttailscale.com/cmd/k8s-operator/operator.go:436\nmain.(*ServiceReconciler).Reconcile\n\ttailscale.com/cmd/k8s-operator/operator.go:327\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\tsigs.k8s.io/controller-runtime@v0.13.1/pkg/internal/controller/controller.go:121\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/controller-runtime@v0.13.1/pkg/internal/controller/controller.go:320\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/controller-runtime@v0.13.1/pkg/internal/controller/controller.go:273\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/controller-runtime@v0.13.1/pkg/internal/controller/controller.go:234"}
{"level":"debug","ts":1680727688.173288,"logger":"service-reconciler","msg":"reconciling statefulset tailscale/ts-jellyseerr-mvt4n","service-ns":"media","service-name":"jellyseerr"}
{"level":"debug","ts":1680727688.186669,"logger":"service-reconciler","msg":"reconcile finished","service-ns":"media","service-name":"jellyseerr"}
{"level":"info","ts":1680727688.186713,"msg":"Observed a panic in reconciler: runtime error: invalid memory address or nil pointer dereference","controller":"service","controllerGroup":"","controllerKind":"Service","Service":{"name":"jellyseerr","namespace":"media"},"namespace":"media","name":"jellyseerr","reconcileID":"6f873a5a-95e1-4009-9c73-1adfce6cfa5d"}
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x110 pc=0x1c51471]
goroutine 1005 [running]:
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile.func1()
sigs.k8s.io/controller-runtime@v0.13.1/pkg/internal/controller/controller.go:118 +0x1f4
panic({0x1e7fc60, 0x3417a50})
runtime/panic.go:884 +0x213
main.(*ServiceReconciler).getDeviceInfo(0xc003620120, {0x249eed0, 0xc003d6d6e0}, 0xc003e03400)
tailscale.com/cmd/k8s-operator/operator.go:569 +0x1f1
main.(*ServiceReconciler).maybeProvision(0xc003620120, {0x249eed0, 0xc003d6d6e0}, 0xc0007c4470?, 0xc003e03400)
tailscale.com/cmd/k8s-operator/operator.go:450 +0x585
main.(*ServiceReconciler).Reconcile(0xc003620120, {0x249eed0, 0xc003d6d6e0}, {{{0xc003481390?, 0x0?}, {0xc003481384?, 0x40e1c7?}}})
tailscale.com/cmd/k8s-operator/operator.go:327 +0x6b8
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0x249ee28?, {0x249eed0?, 0xc003d6d6e0?}, {{{0xc003481390?, 0x205fa80?}, {0xc003481384?, 0x414ca5?}}})
sigs.k8s.io/controller-runtime@v0.13.1/pkg/internal/controller/controller.go:121 +0xc8
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc002c50a00, {0x249ee28, 0xc002c154a0}, {0x1f12f80?, 0xc0033a2400?})
sigs.k8s.io/controller-runtime@v0.13.1/pkg/internal/controller/controller.go:320 +0x309
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc002c50a00, {0x249ee28, 0xc002c154a0})
sigs.k8s.io/controller-runtime@v0.13.1/pkg/internal/controller/controller.go:273 +0x1d9
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2()
sigs.k8s.io/controller-runtime@v0.13.1/pkg/internal/controller/controller.go:234 +0x85
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2
sigs.k8s.io/controller-runtime@v0.13.1/pkg/internal/controller/controller.go:230 +0x587
What are you trying to do?
It's possible to set the hostname of a loadbalancer using
The hostname does not update if the annotation is changed however. The only way to force a refresh is to delete the secret and statefulset associated with the lb.
How should we solve this?
It may be possible to watch for changes to the resource - at a minimum, it should check the hostname when the pod for the statefulset or operator is restarted.
What is the impact of not solving this?
It will be tedious to change the hostname of an lb service.
Anything else?
No response