Using NodePort instead of LoadBalancer service type for Nginx

[mmailhos]

Hello,

Initial problem

I'd like to skip the need of using a load balancer to avoid paying those quiet huge traffic fees from the provider (for a simple non-production application). I decided to start with this attempt on a single node cluster with the Nginx Ingress Controller

Issue

Unable to issue a certificate

Configuration / Environment

I am using the exact same configuration as in examples/nginx directory but changing https://github.com/jetstack/kube-lego/blob/master/examples/nginx/nginx/service.yaml#L7 with NodePort.

I am using GKE . I created a static IP to my single node cluster with a A record on it. Finally I enabled inbound HTTP and HTTPS on the firewall of the instance (GCP).

I also tried with externalIPs. I ended up with the following services:

NAME                                   CLUSTER-IP      EXTERNAL-IP         PORT(S)               AGE
default-http-backend   10.63.253.8        <none>                      80/TCP                   4m
my-app                               10.63.249.202   <none>                     9090/TCP               4m
kube-lego-nginx             10.63.247.119   <none>                     8080/TCP               3m
kubernetes                         10.63.240.1        <none>                     443/TCP                   9m
nginx                                    10.63.247.4       104.1.2.3                  80/TCP,443/TCP   4m

Logs Here is a round of logs from the Lego pod and one from the Nginx Ingress Controller:

In the case of NodePort:

time="2017-06-12T20:52:18Z" level=info msg="ignoring as has no annotation 'kubernetes.io/tls-acme'" context=ingress name=kube-lego-nginx namespace=default 
time="2017-06-12T20:52:18Z" level=info msg="process certificate requests for ingresses" context=kubelego 
time="2017-06-12T20:52:18Z" level=info msg="Attempting to create new secret" context=secret name=myapp-tls namespace=default 
time="2017-06-12T20:52:18Z" level=info msg="no cert associated with ingress" context="ingress_tls" name=myapp namespace=default 
time="2017-06-12T20:52:18Z" level=info msg="requesting certificate for mydomain.com" context="ingress_tls" name=myapp namespace=default 
time="2017-06-12T20:52:18Z" level=info msg="Attempting to create new secret" context=secret name=kube-lego-account namespace=default 

In the case of externalIPs:

time="2017-06-12T22:07:18Z" level=info msg="Attempting to create new secret" context=secret name=my-app-tls namespace=default
time="2017-06-12T22:07:18Z" level=info msg="no cert associated with ingress" context="ingress_tls" name=my-app namespace=default
time="2017-06-12T22:07:18Z" level=info msg="requesting certificate for my-domain.com" context="ingress_tls" name=my-app namespace=default
time="2017-06-12T22:08:31Z" level=warning msg="authorization failed after 1m0s: reachability test failed: Get http://my-domain.com/.well-known/acme-challenge/_selftest: dial tcp 104.199.9.245:80: getsockopt: connection refused" context=acme domain=my-domain.com
time="2017-06-12T22:08:31Z" level=error msg="Error while processing certificate requests: no domain could be authorized successfully" context=kubelego
time="2017-06-12T22:08:31Z" level=info msg="ignoring as has no annotation 'kubernetes.io/tls-acme'" context=ingress name=kube-lego-nginx namespace=default
time="2017-06-12T22:08:31Z" level=info msg="process certificate requests for ingresses" context=kubelego
time="2017-06-12T22:08:31Z" level=info msg="Attempting to create new secret" context=secret name=my-app-tls namespace=default
time="2017-06-12T22:08:31Z" level=info msg="no cert associated with ingress" context="ingress_tls" name=my-app namespace=default
time="2017-06-12T22:08:31Z" level=info msg="requesting certificate for my-domain.com" context="ingress_tls" name=my-app namespace=default

I0612 20:53:20.202731       1 event.go:216] Event(api.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"kube-lego-nginx", UID:"f15ec7fb-4faf-11e7-8e6a-42010a8400e0", APIVersion:"extensions", ResourceVersion:"1997", FieldPath:""}): type: 'Normal' reason: 'UPDATE' default/kube-lego-nginx
W0612 20:53:25.460480       1 controller.go:932] Secret default/myapp-tls does not exists
W0612 20:53:35.492157       1 controller.go:932] Secret default/myapp-tls does not exists
W0612 20:53:45.460819       1 controller.go:932] Secret default/myapp-tls does not exists
I0612 20:53:51.601493       1 event.go:216] Event(api.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"kube-lego-nginx", UID:"f15ec7fb-4faf-11e7-8e6a-42010a8400e0", APIVersion:"extensions", ResourceVersion:"2054", FieldPath:""}): type: 'Normal' reason: 'UPDATE' default/kube-lego-nginx

Beside this specific issue with NodePort, would you have any idea on how to expose my service (called my-app here) without the need of an external load balancer while keeping the kube-lego benefits about certificates management?

Thanks a lot

[munnerz]

So this should be possible using the nginx ingress controller, provided the nginx ingress controller is configured correctly to use NodePort. From looking at your kube-lego logs, specifically time="2017-06-12T22:08:31Z" level=warning msg="authorization failed after 1m0s: reachability test failed: Get http://my-domain.com/.well-known/acme-challenge/_selftest: dial tcp 104.199.9.245:80: getsockopt: connection refused" context=acme domain=my-domain.com, it appears that either the Google Cloud firewall, or instances network settings are refusing the connection to the local nginx-ingress instance.

The official nginx-ingress helm chart supports NodePort as a Service type, so have a go getting set up with that first. Once you can successfully access your service without TLS via the instances IP address, you should be okay to enable kube-lego and all should behave as normal.

[munnerz]

I'm going to close this issue for now, as it appears to be related to the nginx-ingress controller itself and not kube-lego. If you feel this has been closed in haste, please feel free to re-open and ping me!