search

jupyterhub / kubespawner

Kubernetes spawner for JupyterHub
https://jupyterhub-kubespawner.readthedocs.io
BSD 3-Clause "New" or "Revised" License
536 stars 301 forks source link

WIP: Optionally disable k8s service env-var links #750

Closed manics closed 1 month ago

manics commented 1 year ago

By default K8s injects environment variables for every service in the K8s cluster: https://kubernetes.io/docs/tutorials/services/connect-applications-service/#accessing-the-service This is meant to provide a backwards compatible way of accessing K8s services where in-cluster DNS is not available.

E.g. on mybinder.org:

$ env | sort | grep -E _'(SERVICE|PORT)_'

BINDER_PORT_80_TCP_ADDR=10.3.246.84
BINDER_PORT_80_TCP_PORT=80
BINDER_PORT_80_TCP_PROTO=tcp
BINDER_PORT_80_TCP=tcp://10.3.246.84:80
BINDER_SERVICE_HOST=10.3.246.84
BINDER_SERVICE_PORT=80
HUB_PORT_8081_TCP_ADDR=10.3.204.115
HUB_PORT_8081_TCP_PORT=8081
HUB_PORT_8081_TCP_PROTO=tcp
HUB_PORT_8081_TCP=tcp://10.3.204.115:8081
HUB_SERVICE_HOST=10.3.204.115
HUB_SERVICE_PORT=8081
HUB_SERVICE_PORT_HUB=8081
JUPYTERHUB_SERVICE_PREFIX=/user/binder-examples-conda-uhrhszvw/
JUPYTERHUB_SERVICE_URL=http://0.0.0.0:8888/user/binder-examples-conda-uhrhszvw/
KUBERNETES_PORT_443_TCP_ADDR=10.3.0.1
KUBERNETES_PORT_443_TCP_PORT=443
KUBERNETES_PORT_443_TCP_PROTO=tcp
KUBERNETES_PORT_443_TCP=tcp://10.3.0.1:443
KUBERNETES_SERVICE_HOST=10.3.0.1
KUBERNETES_SERVICE_PORT=443
KUBERNETES_SERVICE_PORT_HTTPS=443
OVH2_GRAFANA_PORT_80_TCP_ADDR=10.3.247.169
OVH2_GRAFANA_PORT_80_TCP_PORT=80
OVH2_GRAFANA_PORT_80_TCP_PROTO=tcp
OVH2_GRAFANA_PORT_80_TCP=tcp://10.3.247.169:80
OVH2_GRAFANA_SERVICE_HOST=10.3.247.169
OVH2_GRAFANA_SERVICE_PORT=80
OVH2_GRAFANA_SERVICE_PORT_SERVICE=80
OVH2_INGRESS_NGINX_CONTROLLER_METRICS_PORT_10254_TCP_ADDR=10.3.250.132
OVH2_INGRESS_NGINX_CONTROLLER_METRICS_PORT_10254_TCP_PORT=10254
OVH2_INGRESS_NGINX_CONTROLLER_METRICS_PORT_10254_TCP_PROTO=tcp
OVH2_INGRESS_NGINX_CONTROLLER_METRICS_PORT_10254_TCP=tcp://10.3.250.132:10254
OVH2_INGRESS_NGINX_CONTROLLER_METRICS_SERVICE_HOST=10.3.250.132
OVH2_INGRESS_NGINX_CONTROLLER_METRICS_SERVICE_PORT=10254
OVH2_INGRESS_NGINX_CONTROLLER_METRICS_SERVICE_PORT_METRICS=10254
OVH2_INGRESS_NGINX_CONTROLLER_PORT_443_TCP_ADDR=10.3.235.19
OVH2_INGRESS_NGINX_CONTROLLER_PORT_443_TCP_PORT=443
OVH2_INGRESS_NGINX_CONTROLLER_PORT_443_TCP_PROTO=tcp
OVH2_INGRESS_NGINX_CONTROLLER_PORT_443_TCP=tcp://10.3.235.19:443
OVH2_INGRESS_NGINX_CONTROLLER_PORT_80_TCP_ADDR=10.3.235.19
OVH2_INGRESS_NGINX_CONTROLLER_PORT_80_TCP_PORT=80
OVH2_INGRESS_NGINX_CONTROLLER_PORT_80_TCP_PROTO=tcp
OVH2_INGRESS_NGINX_CONTROLLER_PORT_80_TCP=tcp://10.3.235.19:80
OVH2_INGRESS_NGINX_CONTROLLER_SERVICE_HOST=10.3.235.19
OVH2_INGRESS_NGINX_CONTROLLER_SERVICE_PORT=80
OVH2_INGRESS_NGINX_CONTROLLER_SERVICE_PORT_HTTP=80
OVH2_INGRESS_NGINX_CONTROLLER_SERVICE_PORT_HTTPS=443
OVH2_INGRESS_NGINX_DEFAULTBACKEND_PORT_80_TCP_ADDR=10.3.84.42
OVH2_INGRESS_NGINX_DEFAULTBACKEND_PORT_80_TCP_PORT=80
OVH2_INGRESS_NGINX_DEFAULTBACKEND_PORT_80_TCP_PROTO=tcp
OVH2_INGRESS_NGINX_DEFAULTBACKEND_PORT_80_TCP=tcp://10.3.84.42:80
OVH2_INGRESS_NGINX_DEFAULTBACKEND_SERVICE_HOST=10.3.84.42
OVH2_INGRESS_NGINX_DEFAULTBACKEND_SERVICE_PORT=80
OVH2_INGRESS_NGINX_DEFAULTBACKEND_SERVICE_PORT_HTTP=80
OVH2_KUBE_STATE_METRICS_PORT_8080_TCP_ADDR=10.3.165.12
OVH2_KUBE_STATE_METRICS_PORT_8080_TCP_PORT=8080
OVH2_KUBE_STATE_METRICS_PORT_8080_TCP_PROTO=tcp
OVH2_KUBE_STATE_METRICS_PORT_8080_TCP=tcp://10.3.165.12:8080
OVH2_KUBE_STATE_METRICS_SERVICE_HOST=10.3.165.12
OVH2_KUBE_STATE_METRICS_SERVICE_PORT=8080
OVH2_KUBE_STATE_METRICS_SERVICE_PORT_HTTP=8080
OVH2_PROMETHEUS_NODE_EXPORTER_PORT_9100_TCP_ADDR=10.3.5.41
OVH2_PROMETHEUS_NODE_EXPORTER_PORT_9100_TCP_PORT=9100
OVH2_PROMETHEUS_NODE_EXPORTER_PORT_9100_TCP_PROTO=tcp
OVH2_PROMETHEUS_NODE_EXPORTER_PORT_9100_TCP=tcp://10.3.5.41:9100
OVH2_PROMETHEUS_NODE_EXPORTER_SERVICE_HOST=10.3.5.41
OVH2_PROMETHEUS_NODE_EXPORTER_SERVICE_PORT=9100
OVH2_PROMETHEUS_NODE_EXPORTER_SERVICE_PORT_METRICS=9100
OVH2_PROMETHEUS_SERVER_PORT_80_TCP_ADDR=10.3.110.10
OVH2_PROMETHEUS_SERVER_PORT_80_TCP_PORT=80
OVH2_PROMETHEUS_SERVER_PORT_80_TCP_PROTO=tcp
OVH2_PROMETHEUS_SERVER_PORT_80_TCP=tcp://10.3.110.10:80
OVH2_PROMETHEUS_SERVER_SERVICE_HOST=10.3.110.10
OVH2_PROMETHEUS_SERVER_SERVICE_PORT=80
OVH2_PROMETHEUS_SERVER_SERVICE_PORT_HTTP=80
PROXY_API_PORT_8001_TCP_ADDR=10.3.86.77
PROXY_API_PORT_8001_TCP_PORT=8001
PROXY_API_PORT_8001_TCP_PROTO=tcp
PROXY_API_PORT_8001_TCP=tcp://10.3.86.77:8001
PROXY_API_SERVICE_HOST=10.3.86.77
PROXY_API_SERVICE_PORT=8001
PROXY_PATCHES_PORT_80_TCP_ADDR=10.3.97.135
PROXY_PATCHES_PORT_80_TCP_PORT=80
PROXY_PATCHES_PORT_80_TCP_PROTO=tcp
PROXY_PATCHES_PORT_80_TCP=tcp://10.3.97.135:80
PROXY_PATCHES_SERVICE_HOST=10.3.97.135
PROXY_PATCHES_SERVICE_PORT=80
PROXY_PUBLIC_PORT_80_TCP_ADDR=10.3.186.111
PROXY_PUBLIC_PORT_80_TCP_PORT=80
PROXY_PUBLIC_PORT_80_TCP_PROTO=tcp
PROXY_PUBLIC_PORT_80_TCP=tcp://10.3.186.111:80
PROXY_PUBLIC_SERVICE_HOST=10.3.186.111
PROXY_PUBLIC_SERVICE_PORT=80
PROXY_PUBLIC_SERVICE_PORT_HTTP=80
STATIC_PORT_80_TCP_ADDR=10.3.21.149
STATIC_PORT_80_TCP_PORT=80
STATIC_PORT_80_TCP_PROTO=tcp
STATIC_PORT_80_TCP=tcp://10.3.21.149:80
STATIC_SERVICE_HOST=10.3.21.149
STATIC_SERVICE_PORT=80
manics commented 1 year ago

Example:

$ env | sort | grep -E _'(SERVICE|PORT)_'

JUPYTERHUB_SERVICE_PREFIX=/user/binder-examples-conda-sn3ftxwl/
JUPYTERHUB_SERVICE_URL=http://0.0.0.0:8888/user/binder-examples-conda-sn3ftxwl/
KUBERNETES_PORT_443_TCP_ADDR=172.20.0.1
KUBERNETES_PORT_443_TCP_PORT=443
KUBERNETES_PORT_443_TCP_PROTO=tcp
KUBERNETES_PORT_443_TCP=tcp://172.20.0.1:443
KUBERNETES_SERVICE_HOST=172.20.0.1
KUBERNETES_SERVICE_PORT=443
KUBERNETES_SERVICE_PORT_HTTPS=443

Though I've belatedly realised this can also be set with extra_pod_config. Do you reckon it's worth adding anyway, perhaps as a breaking change where it defaults to False? I can't think of good reason for needing this to be enabled for Jupyter applications.

yuvipanda commented 1 year ago

There's definitely code that relies on the existence of these env vars to 'discover' other services I think. As this could be set with extra_pod_config, IMO we should just let this be.

yuvipanda commented 1 month ago

Closing, as I think we should just let this be. Let me know if you disagree, @manics!