Startup problem with ingress

[braub3rg]

Hello, i not understand how work with ingress,

Setup Ingress with 2 paths:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: http-app-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  defaultBackend:
    service:
      name: service1
      port:
        number: 80
  rules:
  - host: '*.onion'
    http:
      paths:
      - path: /foo/
        pathType: Prefix
        backend:
          service:
            name: service1
            port:
              number: 80
      - path: /bar/
        pathType: Prefix
        backend:
          service:
            name: service2
            port:
              number: 80

And try setup OnionService:

apiVersion: tor.k8s.torproject.org/v1alpha2
kind: OnionService
metadata:
  name: example-onion-service
spec:
  ... (secrets and other) ...
  rules:
    - port:
        number: 80
      backend:
        service:
          # name: service1  # working
          name: http-app-ingress  # not working. What am I doing wrong?
          port:
            number: 80

I try get onion address from CLI, but no result

kubectl get onion
NAME                    HOSTNAME   AGE
example-onion-service              10m

After 10 min tor pod not available (and not working)

[bugfest]

Hi @braub3rg!

Thanks for your bug report :D

TLDR; Set backend.service.name with your ingress controller's service name.

First of all, the example I put in samples is incorrect (my bad). Current example in hack/samples/full-example is publishing the same service both via onion service and ingress, but not the expected: ingress service (and it's rules) exposed as onion. I'll fix it with the manifests below.

The trick is to set the ingress controller address/hostname/fqdn in the backend.service.name. In your example should be something like nginx-ingress-nginx-ingress if you installed nginx ingress with helm using nginx-ingress as the release name. You can find the exact name it by running kubectl get svc in your cluster.

In my test cluster I get:

$ kubectl get svc
NAME                                    TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
kubernetes                              ClusterIP      10.43.0.1      <none>        443/TCP                      70m
nginx-ingress-nginx-ingress             LoadBalancer   10.43.247.30   172.18.0.2    80:32211/TCP,443:32636/TCP   69m
http-app                                ClusterIP      10.43.164.68   <none>        8080/TCP                     65m
...

The steps I followed. From the creation of the test cluster:

k3d cluster create onions --registry-create onions:5000 --k3s-arg "--disable=traefik@server:0"
helm repo add bugfest https://bugfest.github.io/tor-controller
helm repo add nginx-stable https://helm.nginx.com/stable
helm repo update
helm upgrade --install --create-namespace --namespace tor-controller tor-controller bugfest/tor-controller
helm install nginx-ingress nginx-stable/nginx-ingress

And apply these:

# Example: exposing ingress as .onion via OnionService
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: http-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: http-app
  template:
    metadata:
      labels:
        app: http-app
    spec:
      containers:
      - name: http-app
        image: quay.io/bugfest/echoserver:2.5
        ports:
        - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: http-app
  labels:
    app: http-app
spec:
  ports:
  - port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app: http-app
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: http-app-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  # Need to specify the ingress class if you don't have a default one
  # $ kubectl get ingressclass   
  ingressClassName: nginx
  defaultBackend:
    service:
      name: default-http-backend
      port:
        number: 80
  rules:
  - host: '*.onion'
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: http-app
            port:
              number: 8080
---
apiVersion: tor.k8s.torproject.org/v1alpha2
kind: OnionService
metadata:
  name: example-onion-service
spec:
  version: 3
  rules:
    - port:
        number: 80
      backend:
        service:
          # This name will depend on your ingress installation
          # For example, for nginx's ingress installation using helm
          # the name template is [release-name]-nginx-ingress
          #
          # I used this commands:
          # $ helm repo add nginx-stable https://helm.nginx.com/stable
          # $ helm install nginx-ingress nginx-stable/nginx-ingress
          name: nginx-ingress-nginx-ingress
          port:
            number: 80

[bugfest]

@braub3rg I've fixed the example. Please let me know if that solves your issue. If not I'll reopen

[braub3rg]

@bugfest When i use helm for install ingress

$ helm install nginx-ingress nginx-stable/nginx-ingress

And

$ kubectl get services
NAME                          TYPE           CLUSTER-IP     EXTERNAL-IP
kubernetes                    ClusterIP      X.X.X.X      <none>
nginx-ingress-nginx-ingress   LoadBalancer   X.X.X.X   <pending>

i can see nginx-ingress-nginx-ingress and can specify this in OnionService

      backend:
        service:
          name: nginx-ingress-nginx-ingress
          port:
            number: 80

This work for me too, thanks :)

But exist trouble with minikube. If you see docs https://kubernetes.io/docs/tasks/access-application-cluster/ingress-minikube/ . To enable the NGINX Ingress controller, run the following command:

minikube addons enable ingress

And i can't see ingress service name in output command kubectl get services:

$ minikube addons enable ingress
Using image k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.1.1
Using image k8s.gcr.io/ingress-nginx/controller:v1.2.1
Using image k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.1.1
Verifying ingress addon...
The 'ingress' addon is enabled

$ kubectl get services
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   X.X.X.X    <none>        443/TCP   75m

... Which is quite strange

So before I didn't understand what to specify in backend.service.name. I hope my path turned out to be useful to other people as well :)

[bugfest]

Glad to hear that. Thanks for the feedback @braub3rg!

Each ingress implementation will have it's way. I had some issues in the past with minikube ingress addon. I always prefer using a controlled install :sunglasses: