Jupyterhub HTTPS with Letsencrypt + kubernetes external DNS cloudflare

[Jean-Baptiste-Lasselle]

• Jupyterhub HTTPS Letsencrypt: https://z2jh.jupyter.org/en/latest/administrator/security.html#https
• External DNS cloud flare: https://github.com/kubernetes-sigs/external-dns/blob/master/docs/tutorials/cloudflare.md

Oh I stil need to solve one problem before that:

• My Jupyther hub will have an external IP on an IP only reachable from inside the VM
• so i need to setup an IP table to route traffic betwen the external IP assigned to jupyterhub inside the kind cluster, and the actual public IP address... So that will mess up with my external DNS controller, who will add a record to my DNS with the External IP it sees in the kubernetes cluster...: no absolutely not, what i need to do is a reverse-proxy in a container, which is connected to the kind network, and will be able to reach the 172.*.*.* docker network ip ?

Network interfaces oin the VM :

pierre@first-test-vm:~$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp0s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1460 qdisc mq state UP group default qlen 1000
    link/ether 42:01:0a:80:00:02 brd ff:ff:ff:ff:ff:ff
    inet 10.128.0.2/32 brd 10.128.0.2 scope global dynamic enp0s0
       valid_lft 3003sec preferred_lft 3003sec
    inet6 fe80::4001:aff:fe80:2/64 scope link
       valid_lft forever preferred_lft forever
3: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 02:42:72:b9:d7:6b brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:72ff:feb9:d76b/64 scope link
       valid_lft forever preferred_lft forever
5: vethad6c47a@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP group default
    link/ether 02:73:5a:b9:f2:25 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::73:5aff:feb9:f225/64 scope link
       valid_lft forever preferred_lft forever
6: br-ac0439c42dfb: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 02:42:8c:83:ac:f5 brd ff:ff:ff:ff:ff:ff
    inet 172.18.0.1/16 brd 172.18.255.255 scope global br-ac0439c42dfb
       valid_lft forever preferred_lft forever
    inet6 fc00:f853:ccd:e793::1/64 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::42:8cff:fe83:acf5/64 scope link
       valid_lft forever preferred_lft forever
    inet6 fe80::1/64 scope link

hm, hm... yep, there i think my only solution would be to use something more like a DDNS client like https://github.com/decoder-leco/plateforme/issues/5#issue-2317186510 but how can that DDNS client find out about the Ephemeral IP address that GCP assigns to my VM...?

[Jean-Baptiste-Lasselle]

Oh I think I know:

• I need some kind of script which can update my DNS record with a given IP
• I then can first use that script in my terraform with a local exec provisioner, to update my DNS record
• but what if i restart my VM and a new IP is assigned?
  • https://stackoverflow.com/questions/34726498/how-to-update-google-cloud-dns-with-ephemeral-ip-for-an-instance
  • ok the script based on the gcloud cli i can use the part that discovers the VM instance public ephemeral IP
  • but then i would have to run it somewhere that script, like in a kubernetes controller, which would run the script every 3 minutes and if IP cahnged, boum it updates the DNS record in cloudflare....
  • this means i would have to write a kubernetes controller quickly, it is worth ? A static IP would make it enough that i do it with the terraform part...
  • and yet there's still that ip table setup that i would have to do

[Jean-Baptiste-Lasselle]

First test: let's test CLoudflare DDNS

Let's just test without digging too much, if the cloudflare ddns client will be able to discover my public ip...: https://github.com/decoder-leco/plateforme/issues/5

[Jean-Baptiste-Lasselle]

Writing a kubernetes controller quickly :

• This post is a kubebuilder tutorial https://dev.to/ishankhare07/writing-a-simple-kubernetes-controller-in-go-with-kubebuilder-ib8 and there is a long tutorial in the kubebuilder docs, that i should do :
  • https://book.kubebuilder.io/getting-started
  • https://book.kubebuilder.io/cronjob-tutorial/cronjob-tutorial
  • ok and when i have my operator, well it will only change the cloudflare DNS record once i kubectl apply -f my_crd.yaml and the kind in my crd is a new IP addres for my VM like kind: GcpEphemeralIP, and a label or anntation of that CRD will give the Gcp IP address, or whatever, yet i would need a one-2-one association between external IP assigned by metallb and resources of kind: GcpEphemeralIP. Now, the question is, that my job which discovers the ephemeral IP every five minutes would need to run the kubectl apply -f my_crd.yaml. Okay now my operator would econcile like this : as soon as a kind: GcpEphemeralIP resource is updated/created, the operator changes one annotation or label on the jupyterhhub proxy service, annotation or label which needs to be completely ignored by jupyterhub. Well if i want my operator to be dummy, it could just update le label or annotation of any small pod running for itself, i just need my operator to watch some deployment and adjust anything o he's happy about reconciling the state of something, i will find whatever that will be. it could simple by a dummy server giving the public address with an api or on a dummy webpage, whatever.

and there 's this a bit sloppy post : https://itnext.io/kubernetes-custom-controllers-recipes-for-beginners-bbc286c05ef8

[Jean-Baptiste-Lasselle]

ok i checked that i need to add a txt record to my dns, and i need topay on no-ip to be able to add txt record: