jetstack / kube-lego

DEPRECATED: Automatically request certificates for Kubernetes Ingress resources from Let's Encrypt
Apache License 2.0
2.16k stars 267 forks source link

kube-lego SIGSEV segmentation violation when ingress resource is replaced #256

Open Winterflower opened 6 years ago

Winterflower commented 6 years ago

Hi kube-lego team,

steps

  1. Create kube-lego pod using YAML config files in /examples/gce/lego
  2. Create ingress resource with required annotations using the example ingress-tls yaml in /examples/gce/echoserver
  3. I checked the kube-lego pod logs - everything was looking ok and the pod was running
  4. Edit YAML for ingress resource and run kubectl replace --force -f <myingressresourcefile>.yaml

Shortly after this the kube-lego pod started crashing

time="2017-09-12T22:20:23Z" level=info msg="ignoring as has no annotation 'kubernetes.io/tls-acme'" context=ingress name=kube-lego-nginx namespace=kube-lego 
time="2017-09-12T22:20:23Z" level=debug msg=reset context=provider provider=nginx 
time="2017-09-12T22:20:23Z" level=debug msg=finalize context=provider provider=nginx 
time="2017-09-12T22:20:23Z" level=info msg="disable provider no TLS hosts found" context=provider provider=nginx 
time="2017-09-12T22:20:23Z" level=error msg="ingresses.extensions \"kube-lego\" not found" context=provider provider=nginx 
time="2017-09-12T22:20:23Z" level=debug msg=reset context=provider provider=gce 
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x10d4c36]

goroutine 13 [running]:
github.com/jetstack/kube-lego/pkg/provider/gce.(*Gce).Process(0xc4203064e0, 0x1b0dc80, 0xc420163380, 0x3, 0x108b901)
    /go/src/github.com/jetstack/kube-lego/pkg/provider/gce/gce.go:121 +0x1c6
github.com/jetstack/kube-lego/pkg/kubelego.(*KubeLego).processProvider(0xc4201f5680, 0xc4201d3740, 0x1, 0x1, 0x1, 0xc4201d3740)
    /go/src/github.com/jetstack/kube-lego/pkg/kubelego/configure.go:67 +0x1a4
github.com/jetstack/kube-lego/pkg/kubelego.(*KubeLego).reconfigure(0xc4201f5680, 0xc4201633c0, 0x2, 0x2, 0x2, 0x0)
    /go/src/github.com/jetstack/kube-lego/pkg/kubelego/configure.go:96 +0x36c
github.com/jetstack/kube-lego/pkg/kubelego.(*KubeLego).Reconfigure(0xc4201f5680, 0x1392d43, 0x1b)
    /go/src/github.com/jetstack/kube-lego/pkg/kubelego/configure.go:124 +0x8a
github.com/jetstack/kube-lego/pkg/kubelego.(*KubeLego).WatchReconfigure.func2(0xc4201f5680)
    /go/src/github.com/jetstack/kube-lego/pkg/kubelego/watch.go:53 +0x11d
created by github.com/jetstack/kube-lego/pkg/kubelego.(*KubeLego).WatchReconfigure
    /go/src/github.com/jetstack/kube-lego/pkg/kubelego/watch.go:57 +0x88

This was the Ingress resource YAML I used for the --replace

apiVersion: extensions/v1beta1
kind: Ingress
metadata: 
  name: flasktestingress
  annotations: 
    kubernetes.io/tls-acme: "true"
    kubernetes.io/ingress.class: "gce"
spec:
  rules:
   - host: REDACTED
   - http:
      paths:
       - path: /
         backend:
           serviceName: flaskservice
           servicePort: someport
  tls: 
   - secretName: REDACTED
     hosts:
      - REDACTED
Winterflower commented 6 years ago

Today, I've tried further scenarios to try to isolate what is happening:

Scenario 1: Delete all ingresses and deploy kube-lego again. All fine. The pod comes up and does not crash. Add an ingress resource, kube-lego pod starts crashing.

Scenario 2: Add ingress resource, then start kube-lego. Kube-lego pod goes direclty into crash.

rolftimmermans commented 6 years ago

Also running into this. Any ideas for a temporary workaround?