search

caddyserver / ingress

WIP Caddy 2 ingress controller for Kubernetes
Apache License 2.0
638 stars 70 forks source link

Support OnDemandTLS feature #55

Open Sexual opened 3 years ago

Sexual commented 3 years ago

Is it possible to set up an ingress to support all domains?

E.g: host: * rather than host: foo.com

For my use case, I want to support automatic cert issuing for all domains, but the amount of domains is constantly changing and dynamic and can't be manually set in the standard K8s ingress host values.

mholt commented 3 years ago

(I can't answer for the maintainers, but I'll just note, in case it is helpful, that * is not a valid wildcard representation of foo.com -- you'd need either *.com or *.*, which, as certificate subjects, most browsers/clients reject.)

Embraser01 commented 3 years ago

Hi

I think what you search for is On Demand TLS together with a default backend. It is not implemented yet but it's the next thing I want to add 🙂

Sexual commented 3 years ago

@Embraser01 On-Demand TLS with a default backend sounds like just what I need!

Is there any ETA for this or any advice on where to get started to help implement this feature?

Embraser01 commented 3 years ago

I'm looking to implement it before the end of the month, will update here when I've made some progress

Sexual commented 3 years ago

I'm looking to implement it before the end of the month, will update here when I've made some progress

Any update regarding this? Thanks

Embraser01 commented 3 years ago

Any update regarding this? Thanks

Yes! I decided to work on an improved version of the controller and refactored a bunch of things. It also add support for OnDemand TLS.

It's not merged yet as there is still some things to finish but the controller should be working. The code is here and a docker image of the latest commit (this morning) is available here or here.

No documentation yet but it's as easy as adding a few fields in the configmap (JSON schema).

Sexual commented 3 years ago

Any update regarding this? Thanks

Yes! I decided to work on an improved version of the controller and refactored a bunch of things. It also add support for OnDemand TLS.

It's not merged yet as there is still some things to finish but the controller should be working. The code is here and a docker image of the latest commit (this morning) is available here or here.

No documentation yet but it's as easy as adding a few fields in the configmap (JSON schema).

Awesome work! Is this ready to be used in a live environment and are there installation instructions for the improved and refactored ingress?

Embraser01 commented 3 years ago

Awesome work! Is this ready to be used in a live environment and are there installation instructions for the improved and refactored ingress?

The project being still very young, I can't make promises. I can tell you that I've been using it in a live environment for some weeks now and it runs very nicely! I didn't have time to update the documentation yet, but you can check out the chart folder and use helm to generate your manifests https://github.com/caddyserver/ingress/tree/66c52c682f497022f43ffb529def89c3a8ff3472/charts/caddy-ingress-controller

Sexual commented 3 years ago

Awesome work! Is this ready to be used in a live environment and are there installation instructions for the improved and refactored ingress?

The project being still very young, I can't make promises. I can tell you that I've been using it in a live environment for some weeks now and it runs very nicely! I didn't have time to update the documentation yet, but you can check out the chart folder and use helm to generate your manifests https://github.com/caddyserver/ingress/tree/66c52c682f497022f43ffb529def89c3a8ff3472/charts/caddy-ingress-controller

Do you have any insight into how to configure the default backend? I've tried deploying it and checking the logs, it's constantly checking an existing ingress' hosts and trying to issue certificates for them.

Update: I've deployed the updated chart (pr-60 image tag) and I can point a domain to the IP and it loads using HTTP, but does not connect via HTTPS, just gets a standard SSL_ERR.

I've also configured an ingress with the caddy ingress class and it crashes instantly:

[caddy-caddy-ingress-controller-678dcbb84f-v8rln] {"level":"info","ts":1608503299.0471146,"caller":"controller/action_ingress.go:46","msg":"Ingress created (default/caddy-ingress)"} 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] E1220 22:28:22.173607       1 runtime.go:78] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] E1220 22:28:19.047582       1 runtime.go:78] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] goroutine 66 [running]: 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] goroutine 81 [running]: 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] k8s.io/apimachinery/pkg/util/runtime.logPanic(0x1c5b760, 0x32e81d0) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] k8s.io/apimachinery/pkg/util/runtime.logPanic(0x1c5b760, 0x32e81d0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/runtime/runtime.go:74 +0xa3 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/runtime/runtime.go:74 +0xa3 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/runtime/runtime.go:48 +0x82 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/runtime/runtime.go:48 +0x82 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] panic(0x1c5b760, 0x32e81d0) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] panic(0x1c5b760, 0x32e81d0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /usr/local/go/src/runtime/panic.go:969 +0x166 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /usr/local/go/src/runtime/panic.go:969 +0x166 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] github.com/caddyserver/ingress/internal/caddy.LoadIngressConfig(0xc000144320, 0xc0004fe390, 0xc0004fe390, 0xc000144320) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] github.com/caddyserver/ingress/internal/caddy.LoadIngressConfig(0xc0005981e0, 0xc0004f42d0, 0xc0004f42d0, 0xc0005981e0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /app/internal/caddy/ingress.go:59 +0x56f 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /app/internal/caddy/ingress.go:59 +0x56f 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] github.com/caddyserver/ingress/internal/caddy.Converter.ConvertToCaddyConfig(0xc00005400e, 0x7, 0xc0004fe390, 0x0, 0x1, 0xc00020cbd0, 0x0) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] github.com/caddyserver/ingress/internal/caddy.Converter.ConvertToCaddyConfig(0xc00005400e, 0x7, 0xc0004f42d0, 0x0, 0x1, 0xc00000e090, 0x0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /app/internal/caddy/convert.go:91 +0x60 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /app/internal/caddy/convert.go:91 +0x60 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] github.com/caddyserver/ingress/internal/controller.(*CaddyController).reloadCaddy(0xc0001faa80, 0xc0001faa80, 0x0) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] github.com/caddyserver/ingress/internal/controller.(*CaddyController).reloadCaddy(0xc000268690, 0xc000268690, 0x0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /app/internal/controller/controller.go:262 +0x67 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /app/internal/controller/controller.go:262 +0x67 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] github.com/caddyserver/ingress/internal/controller.(*CaddyController).processNextItem(0xc0001faa80, 0x203000) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] github.com/caddyserver/ingress/internal/controller.(*CaddyController).processNextItem(0xc000268690, 0x203000) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /app/internal/controller/controller.go:246 +0x122 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /app/internal/controller/controller.go:246 +0x122 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] github.com/caddyserver/ingress/internal/controller.(*CaddyController).runWorker(0xc0001faa80) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] github.com/caddyserver/ingress/internal/controller.(*CaddyController).runWorker(0xc000268690) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /app/internal/controller/controller.go:222 +0x2b 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc0003b6040) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /app/internal/controller/controller.go:222 +0x2b 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:155 +0x5f 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc00011b490) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc0003b6040, 0x2279ee0, 0xc000488240, 0x1, 0xc00053e360) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:155 +0x5f 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:156 +0xa3 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc00011b490, 0x2279ee0, 0xc00037f2f0, 0x1, 0xc0000966c0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc0003b6040, 0x3b9aca00, 0x0, 0x2002c01, 0xc00053e360) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:156 +0xa3 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:133 +0x98 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc00011b490, 0x3b9aca00, 0x0, 0xc000418101, 0xc0000966c0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] k8s.io/apimachinery/pkg/util/wait.Until(0xc0003b6040, 0x3b9aca00, 0xc00053e360) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:133 +0x98 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:90 +0x4d 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] k8s.io/apimachinery/pkg/util/wait.Until(0xc00011b490, 0x3b9aca00, 0xc0000966c0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] created by github.com/caddyserver/ingress/internal/controller.(*CaddyController).Run 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:90 +0x4d 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /app/internal/controller/controller.go:201 +0x244 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] created by github.com/caddyserver/ingress/internal/controller.(*CaddyController).Run 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] panic: runtime error: invalid memory address or nil pointer dereference [recovered] 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /app/internal/controller/controller.go:201 +0x244 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       panic: runtime error: invalid memory address or nil pointer dereference 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] panic: runtime error: invalid memory address or nil pointer dereference [recovered] 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1a7030f] 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       panic: runtime error: invalid memory address or nil pointer dereference 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]  
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1a7030f] 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] goroutine 66 [running]: 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]  
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] goroutine 81 [running]: 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/runtime/runtime.go:55 +0x105 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] panic(0x1c5b760, 0x32e81d0) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/runtime/runtime.go:55 +0x105 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /usr/local/go/src/runtime/panic.go:969 +0x166 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] panic(0x1c5b760, 0x32e81d0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] github.com/caddyserver/ingress/internal/caddy.LoadIngressConfig(0xc000144320, 0xc0004fe390, 0xc0004fe390, 0xc000144320) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /usr/local/go/src/runtime/panic.go:969 +0x166 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /app/internal/caddy/ingress.go:59 +0x56f 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] github.com/caddyserver/ingress/internal/caddy.LoadIngressConfig(0xc0005981e0, 0xc0004f42d0, 0xc0004f42d0, 0xc0005981e0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] github.com/caddyserver/ingress/internal/caddy.Converter.ConvertToCaddyConfig(0xc00005400e, 0x7, 0xc0004fe390, 0x0, 0x1, 0xc00020cbd0, 0x0) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /app/internal/caddy/ingress.go:59 +0x56f 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /app/internal/caddy/convert.go:91 +0x60 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] github.com/caddyserver/ingress/internal/caddy.Converter.ConvertToCaddyConfig(0xc00005400e, 0x7, 0xc0004f42d0, 0x0, 0x1, 0xc00000e090, 0x0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] github.com/caddyserver/ingress/internal/controller.(*CaddyController).reloadCaddy(0xc0001faa80, 0xc0001faa80, 0x0) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /app/internal/caddy/convert.go:91 +0x60 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /app/internal/controller/controller.go:262 +0x67 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] github.com/caddyserver/ingress/internal/controller.(*CaddyController).reloadCaddy(0xc000268690, 0xc000268690, 0x0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] github.com/caddyserver/ingress/internal/controller.(*CaddyController).processNextItem(0xc0001faa80, 0x203000) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /app/internal/controller/controller.go:262 +0x67 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /app/internal/controller/controller.go:246 +0x122 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] github.com/caddyserver/ingress/internal/controller.(*CaddyController).processNextItem(0xc000268690, 0x203000) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] github.com/caddyserver/ingress/internal/controller.(*CaddyController).runWorker(0xc0001faa80) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /app/internal/controller/controller.go:246 +0x122 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /app/internal/controller/controller.go:222 +0x2b 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] github.com/caddyserver/ingress/internal/controller.(*CaddyController).runWorker(0xc000268690) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc0003b6040) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /app/internal/controller/controller.go:222 +0x2b 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:155 +0x5f 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc00011b490) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc0003b6040, 0x2279ee0, 0xc000488240, 0x1, 0xc00053e360) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:155 +0x5f 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:156 +0xa3 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc00011b490, 0x2279ee0, 0xc00037f2f0, 0x1, 0xc0000966c0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc0003b6040, 0x3b9aca00, 0x0, 0x2002c01, 0xc00053e360) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:156 +0xa3 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:133 +0x98 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc00011b490, 0x3b9aca00, 0x0, 0xc000418101, 0xc0000966c0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] k8s.io/apimachinery/pkg/util/wait.Until(0xc0003b6040, 0x3b9aca00, 0xc00053e360) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:133 +0x98 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:90 +0x4d 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] k8s.io/apimachinery/pkg/util/wait.Until(0xc00011b490, 0x3b9aca00, 0xc0000966c0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] created by github.com/caddyserver/ingress/internal/controller.(*CaddyController).Run 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:90 +0x4d 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /app/internal/controller/controller.go:201 +0x244 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] created by github.com/caddyserver/ingress/internal/controller.(*CaddyController).Run 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /app/internal/controller/controller.go:201 +0x244

Ingress example:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: caddy-ingress
  annotations:
    kubernetes.io/ingress.class: "caddy"
spec:
  rules:
    - host: "testing.replaced-domain.com"
      http:
        paths:
          - path: /
            backend:
              serviceName: backend
              servicePort: 3000

This error seems to be due to K8s 1.16 not supporting spec.rules.http.paths.pathType? Any ideas?

UPDATE 2: Removing the ingress path fixes the crash. Now my only issue is simply getting the default backend implemented. It's supported in K8s 1.19, but is there a way of implementing it in the same manner as nginx-ingress? https://kubernetes.github.io/ingress-nginx/user-guide/default-backend/

Trying to remove the spec.rules.http.host value (which I thought would lead to all hosts being matched) results in the request just being served by a blank page by caddy and a SSL_ERR again if via HTTPS

Sexual commented 3 years ago

@Embraser01 So after a lot of experimenting, I've gotten a lot further on this.

  1. Having a wildcard for ALL hosts supporting automatic HTTPS isn't possible as the hostnames must be explicitly set: https://caddyserver.com/docs/automatic-https#activation

Any of the following will prevent automatic HTTPS from being activated, either in whole or in part: Not providing any hostnames or IP addresses in the config

  1. The proxyProtocol is not working preventing actual production use. When trying to access https://example.com (a domain configured in the ingress and pointed to the external load balancer IP). It logs the following errors when trying to access via HTTPS: 
    [caddy-caddy-ingress-controller-7b5c7c9577-z7r8b] {"level":"debug","ts":1608994766.4413562,"logger":"http.stdlib","msg":"http: TLS handshake error from 10.128.0.61:8846: invalid signature"} 
[caddy-caddy-ingress-controller-7b5c7c9577-z7r8b] {"level":"debug","ts":1608994766.6447232,"logger":"http.stdlib","msg":"http: TLS handshake error from 10.128.0.61:22483: invalid signature"} 
[caddy-caddy-ingress-controller-7b5c7c9577-vt5qw] {"level":"debug","ts":1608994766.6778147,"logger":"http.stdlib","msg":"http: TLS handshake error from 10.36.5.1:16605: invalid signature"} 
[caddy-caddy-ingress-controller-7b5c7c9577-z7r8b] {"level":"debug","ts":1608994766.8814533,"logger":"http.stdlib","msg":"http: TLS handshake error from 10.36.4.1:59504: invalid signature"} 
[caddy-caddy-ingress-controller-7b5c7c9577-z7r8b] {"level":"debug","ts":1608994767.8734257,"logger":"http.stdlib","msg":"http: TLS handshake error from 10.36.4.1:31946: invalid signature"} 
[caddy-caddy-ingress-controller-7b5c7c9577-z7r8b] {"level":"debug","ts":1608994768.0778928,"logger":"http.stdlib","msg":"http: TLS handshake error from 10.128.0.61:11642: invalid signature"} 
[caddy-caddy-ingress-controller-7b5c7c9577-z7r8b] {"level":"debug","ts":1608994768.1217213,"logger":"http.stdlib","msg":"http: TLS handshake error from 10.128.0.61:5770: invalid signature"} 
[caddy-caddy-ingress-controller-7b5c7c9577-vt5qw] {"level":"debug","ts":1608994768.3253047,"logger":"http.stdlib","msg":"http: TLS handshake error from 10.36.5.1:57416: invalid signature"}

Checking kubectl get pod -o wide, only 10.128.0.61 is visible in the list of IPs and strangely is GKE's monitoring-prometheus-node-exporter

When trying to access via HTTP, it provides a white error page with the text: 400 Bad Request

Embraser01 commented 3 years ago
  1. Having a wildcard for ALL hosts supporting automatic HTTPS isn't possible as the hostnames must be explicitly set: caddyserver.com/docs/automatic-https#activation

Could you provide an ingress .yml and the configmap .yml files that would help me reproduce your issue? And if you also can provide us the applied config.json (it's logged by the controller).

  1. The proxyProtocol is not working preventing actual production use. When trying to access https://example.com (a domain configured in the ingress and pointed to the external load balancer IP). It logs the following errors when trying to access via HTTPS:

For now, when PROXY protocol is enabled, it prevent any connection that do not use PROXY protocol. I don't know exactly your setup but I know that in order to enable PROXY Protocol in AWS, I had to make sure the load balancer in front of Caddy is in ip-mode.

With ip mode enabled: LoadBalancer -> Caddy Ingress Controller Pod Without it: LoadBalancer -> Node -> Caddy Ingress Controller Pod

kyranb commented 3 years ago

Just confirming, thanks to #65 will on demand TLS for any host name be possible? As is currently possible when using Caddy standalone (not as an ingress).

Sexual commented 3 years ago

@Embraser01 It appears that this is due to the PROXY protocol not being something that is explicitly able to be set on GKE. https://projectcontour.io/guides/proxy-proto/

I'm not too sure how this works with ingress-nginx though as I have no issues with that.

Is there any workarounds to this issue?

RickFoland commented 2 years ago

I'd also like to confirm that OnDemand is supposed to be working. I spun up the ingress controller with --set ingressController.config.onDemandTLS=true and although it issues certs as expected, it serves a blank page instead of following the backend specified in my ingress resource.