ARM - [dumb-init] -nginx-plus=false: No such file or directory

[GraemeMitchell84]

I'm trying to setup nginx ingress on a k8s environment with docker running on Raspberry Pi 4 for the first time. I've read through some of the other issues regarding ARM support and landed on the following: -

https://github.com/kubernetes/ingress-nginx/issues/4876

This suggests installation via helm something like this (I just updated to the latest version tag): -

helm install nginx-ingress nginx-stable/nginx-ingress --set controller.image.repository=quay.io/kubernetes-ingress-controller/nginx-ingress-controller-arm --set controller.image.tag="0.32.0"

Helm install shows success, but the nginx ingress pod container is constantly failing. When inspecting logs I see:-

[dumb-init] -nginx-plus=false: No such file or directory

I had a look at the docker file and see the entrypoint is /usr/bin/dumb-init. I spun up a container using the quay image and tried running the following and saw the same error: -

 ./dumb-init -- -nginx-plus=false -nginx-configmaps=default/nginx-ingress-nginx-ingress -default-server-tls-secret=default/nginx-ingress-nginx-ingress-default-server-secret -ingress-class=nginx -use-ingress-class-only=false -health-status=false -health-status-uri=/nginx-health -nginx-debug=false -v=1 -nginx-status=true -nginx-status-port=8080 -nginx-status-allow-cidrs=127.0.0.1 -report-ingress-status -external-service=nginx-ingress-nginx-ingress -enable-leader-election=true -leader-election-lock-name=nginx-ingress-nginx-ingress-leader-election -enable-prometheus-metrics=true -prometheus-metrics-listen-port=9113 -enable-custom-resources=true -enable-tls-passthrough=false quay.io/kubernetes-ingress-controller/nginx-ingress-controller-arm:0.32.0

Here's the description of the pod created for ingress which I used to build the above: -

Name:         nginx-ingress-nginx-ingress-fb7c948b4-spmlp
Namespace:    default
Priority:     0
Node:         k8s-worker-3/10.0.0.204
Start Time:   Mon, 18 May 2020 14:05:59 +0100
Labels:       app=nginx-ingress-nginx-ingress
              pod-template-hash=fb7c948b4
Annotations:  prometheus.io/port: 9113
              prometheus.io/scrape: true
Status:       Running
IP:           10.44.0.0
IPs:
  IP:           10.44.0.0
Controlled By:  ReplicaSet/nginx-ingress-nginx-ingress-fb7c948b4
Containers:
  nginx-ingress-nginx-ingress:
    Container ID:  docker://b6f6f1762ebe34934aa1214ddd4ff5ffba80426dae26a18b09e169f26390b964
    Image:         quay.io/kubernetes-ingress-controller/nginx-ingress-controller-arm:0.32.0
    Image ID:      docker-pullable://quay.io/kubernetes-ingress-controller/nginx-ingress-controller-arm@sha256:9f61cdb1cd1dd720be2ef1a69002ad92cd5dc25302c36cc806ff36889d5a8f6d
    Ports:         80/TCP, 443/TCP, 9113/TCP
    Host Ports:    0/TCP, 0/TCP, 0/TCP
    Args:
      -nginx-plus=false
      -nginx-configmaps=$(POD_NAMESPACE)/nginx-ingress-nginx-ingress
      -default-server-tls-secret=$(POD_NAMESPACE)/nginx-ingress-nginx-ingress-default-server-secret
      -ingress-class=nginx
      -use-ingress-class-only=false
      -health-status=false
      -health-status-uri=/nginx-health
      -nginx-debug=false
      -v=1
      -nginx-status=true
      -nginx-status-port=8080
      -nginx-status-allow-cidrs=127.0.0.1
      -report-ingress-status
      -external-service=nginx-ingress-nginx-ingress
      -enable-leader-election=true
      -leader-election-lock-name=nginx-ingress-nginx-ingress-leader-election
      -enable-prometheus-metrics=true
      -prometheus-metrics-listen-port=9113
      -enable-custom-resources=true
      -enable-tls-passthrough=false
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    2
      Started:      Mon, 18 May 2020 14:32:16 +0100
      Finished:     Mon, 18 May 2020 14:32:16 +0100
    Ready:          False
    Restart Count:  10
    Environment:
      POD_NAMESPACE:  default (v1:metadata.namespace)
      POD_NAME:       nginx-ingress-nginx-ingress-fb7c948b4-spmlp (v1:metadata.name)
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from nginx-ingress-nginx-ingress-token-542bm (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  nginx-ingress-nginx-ingress-token-542bm:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  nginx-ingress-nginx-ingress-token-542bm
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s

Events:

  Type     Reason     Age                  From                   Message
  ----     ------     ----                 ----                   -------
  Normal   Scheduled  <unknown>            default-scheduler      Successfully assigned default/nginx-ingress-nginx-ingress-fb7c948b4-spmlp to k8s-worker-3
  Normal   Pulled     29m (x5 over 31m)    kubelet, k8s-worker-3  Container image "quay.io/kubernetes-ingress-controller/nginx-ingress-controller-arm:0.32.0" already present on machine
  Normal   Created    29m (x5 over 31m)    kubelet, k8s-worker-3  Created container nginx-ingress-nginx-ingress
  Normal   Started    29m (x5 over 31m)    kubelet, k8s-worker-3  Started container nginx-ingress-nginx-ingress
  Warning  BackOff    76s (x140 over 31m)  kubelet, k8s-worker-3  Back-off restarting failed container

Versions: -

Kubernetes Server: v1.18.2
Kubernetes Client: v1.18.2
Docker CE Server: 19.03.8
Docker CE Client: 19.03.8
Raspbian Buster
Linux k8s-worker-3 4.19.97-v7l+ #1294 SMP Thu Jan 30 13:21:14 GMT 2020 armv7l GNU/Linux

Am I doing something wrong here?

[aledbf]

-nginx-plus=false

Closing. You are mixing flags from two different ingress controller. Please remove that flag. This are the valid flags https://kubernetes.github.io/ingress-nginx/user-guide/cli-arguments/

[GraemeMitchell84]

Thanks for the document, but the point is that I didn't set this property. The ingress controller was installed via helm (exact command I ran is in initial post for ref), so based on this feedback it would mean you're implying the helm chart is wrong wouldn't it?

Would your recommendation therefore be that I raise this as an issue against the helm chart @ https://github.com/helm/charts/tree/master/stable/nginx-ingress

As a matter of idle curiosity I ran the following to override that argument to true as per https://docs.nginx.com/nginx-ingress-controller/installation/installation-with-helm/ : -

helm install nginx-ingress nginx-stable/nginx-ingress --set controller.image.repository=quay.io/kubernetes-ingress-controller/nginx-ingress-controller-arm --set controller.image.tag="0.32.0" --set controller.nginxplus=true

Same error again as would be expected of course, but value reporting true rather than false of course: -

[dumb-init] -nginx-plus=true: No such file or directory

[aledbf]

@GraemeMitchell84 like I mentioned, you are mixing ingress controllers.

https://docs.nginx.com/nginx-ingress-controller/installation/installation-with-helm/ : -

That URL is for a different ingress controller.

Would your recommendation therefore be that I raise this as an issue against the helm chart @ https://github.com/helm/charts/tree/master/stable/nginx-ingress

This is the correct chart for this project that now is being migrated here https://kubernetes.github.io/ingress-nginx/deploy/#using-helm

[GraemeMitchell84]

Ah, the penny has dropped as I misinterpreted the "two different ingress controllers" comment! I never knew there were two different projects. So in summary, I was using the helm chart from the other ingress controller project and then overriding the image (to try and get the ARM version) to be the one from this repo.

In my defence - I copied the helm command from a comment in issue #4876 that someone said was working. I got this working following the docs you provided and overriding the image to the ARM one. Thanks @aledbf