502 Bad Gateway bij V5 - Githubissues

kooskaspers commented 2 years ago

Support guidelines

[X] I've read the support guidelines

I've found an issue and checked that ...

[X] ... the documentation does not mention anything about my problem
[X] ... there are no open or closed issues that are related to my problem

Description

Image geupgrade van amd64-2022.01.04 naar amd64-2022.02.03 (dus van v4 naar v5). Krijg echter de webinterface (webservice) niet meer werkend. Logs: 2022/02/22 18:40:17 [crit] 296#296: *1 connect() to unix:///var/tmp/gunicorn--dsmr_webinterface.socket failed (2: No such file or directory) while connecting to upstream, client: 10.42.1.118, server: _, request: "GET / HTTP/1.1", upstream: "http://unix:///var/tmp/gunicorn--dsmr_webinterface.socket:/", "dsmr.domotics.mijndomein.nl"

Expected behaviour

Werkende webinterface

Actual behaviour

Zie bovengenoemde error.

Steps to reproduce

Image geupgrade van amd64-2022.01.04 naar amd64-2022.02.03

Docker info

Kubernetes k3os version v0.21.5-k3s2r1

Version

Ik gebruik Kubernetes op k3os: Linux kubernetes 5.4.0-88-generic #99 SMP Tue Oct 5 16:53:38 UTC 2021 x86_64 GNU/Linux

Docker compose

Geen dc.

Container logs

Zie boven.

Additional info

No response

dennissiemensma commented 2 years ago

Puur ter info:

 unix:///var/tmp/

Dat pad is in v5 aangepast naar:

 unix:///tmp/

xirixiz commented 2 years ago

Mogelijk moet je de image even verwijderen en opnieuw binnen halen. Het pad is namelijk een tijd geleden al aangepast: De image heb ik zojuist nog een keer gecheckt en daarin is de config correct voor de release 2022.02.03.

https://github.com/xirixiz/dsmr-reader-docker/blob/master/rootfs/etc/services.d/dsmr_webinterface/run#L6

Starting DSMR Reader - nginx...
[2022-02-22 21:02:40 +0100] [304] [INFO] Starting gunicorn 20.1.0
[2022-02-22 21:02:40 +0100] [304] [INFO] Listening at: unix:/tmp/gunicorn--dsmr_webinterface.socket (304)
[2022-02-22 21:02:40 +0100] [304] [INFO] Using worker: sync

kooskaspers commented 2 years ago

Nogmaals getest, wederom met de laatste image, zie ook de k8s deployment:

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: dsmr-local
  namespace: domotics
  annotations:
    kubernetes.io/ingress.class: "nginx"    
spec:
  rules:
  - host: dsmr.lan
    http:
      paths:
        - pathType: Prefix
          path: /
          backend:
            service:
              name: dsmr
              port: 
                number: 80
---
apiVersion: v1
kind: Service
metadata:
  name: dsmr
  namespace: domotics
  labels:
    app: dsmr
spec:
  ports:
      # Expose the service on the specified port internally within the cluster. 
    - port: 80
      name: "http"
      protocol: TCP
  selector:
    app: dsmr
    tier: frontend
  clusterIP: None #headless, want ingress
---
apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
kind: DaemonSet
metadata:
  name: dsmr
  namespace: domotics
  labels:
    app: dsmr
spec:
  selector:
    matchLabels:
      app: dsmr
      tier: frontend
  #dit is het pod template:
  template:
    metadata:
      labels:
        app: dsmr
        tier: frontend
    spec:
      nodeName: kubernetes
      containers:
      - env:
        - name: DJANGO_DATABASE_HOST
          value: dsmrdb.domotics
        - name: ENABLE_IFRAME
          value: "true"
        - name: DSMRREADER_ADMIN_USER
          value: admin
        - name: DSMRREADER_ADMIN_PASSWORD
          valueFrom:
            secretKeyRef:
              name: dsmr-admin-password
              key: password
        # https://hub.docker.com/r/xirixiz/dsmr-reader-docker/tags
        # changelog: https://dsmr-reader.readthedocs.io/en/v5/reference/changelog.html
        image: xirixiz/dsmr-reader-docker:amd64-2022.02.03
        name: dsmr
        ports:
        - containerPort: 80
        - containerPort: 443
        resources: {}
        securityContext:
          privileged: true #anders geen toegang tot usb
          capabilities:
            add:
            - NET_ADMIN
        volumeMounts:
        - mountPath: /etc/nginx/conf.d/
          name: dsmr-storage
        - mountPath: /dsmr/backups-by-dsmr
          name: dsmr-backups
        - mountPath: /dev/ttyUSB0
          name: dev-usb0
      restartPolicy: Always
      volumes:
      - name: dsmr-storage
        hostPath:
          path: /var/lib/rancher/k3s/storage/domotics/dsmr
      - name: dsmr-backups
        hostPath:
          path: /var/lib/rancher/k3s/storage/domotics/dsmr-backups
      - name: dev-usb0
        hostPath:
          path: /dev/ttyUSB0

Ik twijfel er in ieder geval niet aan dat ik niet gebruik maak van de laatste image, door deze expliciet te specificeren.

Volledige logging:

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 10-adduser: executing... 
usermod: no changes

-------------------------
User UID: 803
User GID: 803
-------------------------

[cont-init.d] 10-adduser: exited 0.
[cont-init.d] 20-set-app-defaults: executing... 

[ [00;34mINFO[0m ] DSMR Reader release: 5.0.0

[ [00;34mINFO[0m ] DSMR Reader Docker release: 2022.02.03

[ [00;34mINFO[0m ] Ensure permissions...

[ [00;34mINFO[0m ] Setting architecture requirements...

[ [00;34mINFO[0m ] X64 Architecture

[ [00;34mINFO[0m ] Enabling IFrame...

[ [00;34mINFO[0m ] Fixing /dev/ttyUSB* security...

[ [00;34mINFO[0m ] Verifying if the DSMR web credential variables have been set...

[ [00;34mINFO[0m ] Verifying database connectivity to host: dsmrdb.domotics with port: 5432...

[ [00;34mINFO[0m ] Database connectivity successfully verified!

[ [00;34mINFO[0m ] Running post configuration...
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, dsmr_api, dsmr_backend, dsmr_backup, dsmr_consumption, dsmr_datalogger, dsmr_dropbox, dsmr_frontend, dsmr_influxdb, dsmr_mindergas, dsmr_mqtt, dsmr_notification, dsmr_pvoutput, dsmr_stats, dsmr_weather, sessions
Running migrations:
  No migrations to apply.

559 static files copied to '/var/www/dsmrreader/static'.
Updating password of superuser "admin"
Deactivating any other existing superusers

[ [00;34mINFO[0m ] Checking for NGINX SSL configuration...

[ [00;34mINFO[0m ] ENABLE_NGINX_SSL is disabled, nothing to see here. Continuing...

[ [00;34mINFO[0m ] Checking for HTTP AUTHENTICATION configuration...

[ [00;34mINFO[0m ] ENABLE_HTTP_AUTH is disabled, nothing to see here. Continuing...

[ [00;34mINFO[0m ] Configuring DSMR in standlone datalogger mode....
[cont-init.d] 20-set-app-defaults: exited 0.
[cont-init.d] done.
[services.d] starting services
Starting DSMR Reader - backend...
Starting DSMR Reader - nginx...
Starting DSMR Reader - datalogger...
Starting DSMR Reader - webinterface...
[services.d] done.
[2022-02-22 22:21:17 +0000] [274] [INFO] Starting gunicorn 20.1.0
[2022-02-22 22:21:17 +0000] [274] [INFO] Listening at: unix:/tmp/gunicorn--dsmr_webinterface.socket (274)
[2022-02-22 22:21:17 +0000] [274] [INFO] Using worker: sync
[2022-02-22 22:21:17 +0000] [299] [INFO] Booting worker with pid: 299
Current logging level set to "ERROR". More information can be found here: https://dsmr-reader.readthedocs.io/en/latest/how-to/troubleshooting/enabling-debug-logging.html
Current logging level set to "ERROR". More information can be found here: https://dsmr-reader.readthedocs.io/en/latest/how-to/troubleshooting/enabling-debug-logging.html
2022/02/22 22:25:55 [crit] 289#289: *68 connect() to unix:///var/tmp/gunicorn--dsmr_webinterface.socket failed (2: No such file or directory) while connecting to upstream, client: 10.42.1.118, server: _, request: "GET / HTTP/1.1", upstream: "http://unix:///var/tmp/gunicorn--dsmr_webinterface.socket:/", host: "dsmr.lan"
10.42.1.118 - - [22/Feb/2022:22:25:55 +0000] "GET / HTTP/1.1" 502 150 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:97.0) Gecko/20100101 Firefox/97.0" "10.42.1.1"

Hier lijkt het overigens wel goed te gaan: [2022-02-22 22:21:17 +0000] [274] [INFO] Listening at: unix:/tmp/gunicorn--dsmr_webinterface.socket (274)

Totdat ik een http request doe:

reader.readthedocs.io/en/latest/how-to/troubleshooting/enabling-debug-logging.html
2022/02/22 22:25:55 [crit] 289#289: *68 connect() to unix:///var/tmp/gunicorn--dsmr_webinterface.socket failed (2: No such file or directory) while connecting to upstream, client: 10.42.1.118, server: _, request: "GET / HTTP/1.1", upstream: "http://unix:///var/tmp/gunicorn--dsmr_webinterface.socket:/", host: "dsmr.lan"
10.42.1.118 - - [22/Feb/2022:22:25:55 +0000] "GET / HTTP/1.1" 502 150 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:97.0) Gecko/20100101 Firefox/97.0" "10.42.1.1"

xirixiz commented 2 years ago

Bedankt voor het delen van de extra details! Helaas heb ik de post gemist maar ik zie wel waar het mis gaat. Je mount mountPath: /etc/nginx/conf.d/. De http(s) requests lopen via nginx dus ik vermoed dat daar nog een oude verwijzing staat. Is er een reden dat je zelf een andere config mount?

Update: zie idd in regel 2 in de nginx config een verwijzing naar unix:///var/tmp/gunicorn.....

Ik weet zeker dat dit het issue zal oplossen dus ik sluit dit issue ook meteen :). Mocht het nodig zijn dan kan je het natuurlijk weer heropenen.

xirixiz / dsmr-reader-docker

502 Bad Gateway bij V5 #270

Support guidelines

I've found an issue and checked that ...

Description

Expected behaviour

Actual behaviour

Steps to reproduce

Docker info

Version

Docker compose

Container logs

Additional info