Webinterface not starting on 2024.02.03

daxy01 commented 4 months ago

Support guidelines

[X] I've read the support guidelines

I've found an issue and checked that ...

[X] ... the documentation does not mention anything about my problem
[X] ... there are no open or closed issues that are related to my problem

Description

After upgrading to 2024.02.03, dsmr-reader won't start anymore and container is "unhealthy". When (re-)using 5.10.4-2023.11.01 all works instantly.

Expected behaviour

UI should start.

Actual behaviour

WebUI is not serving data, Chrome tells me "ERR_CONNECTION_REFUSED"

Steps to reproduce

"docker-compose pull" to get 2024.02.03
"docker-compose up -d" to start 2024.02.03
"docker ps" to see container is unhealthy

Docker info

Client: Docker Engine - Community
 Version:    25.0.3
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.12.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.24.5
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 39
  Running: 30
  Paused: 0
  Stopped: 9
 Images: 62
 Server Version: 25.0.3
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.1.0-18-amd64
 Operating System: Debian GNU/Linux 12 (bookworm)
 OSType: linux
 Architecture: x86_64
 CPUs: 12
 Total Memory: 62.51GiB
 Name: orion
 ID: ZF3H:LWJT:ZHWU:D6PY:5CLN:6B5F:WEJL:PDE7:BDKG:Y5SX:CGBY:7B6G
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: true
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Version

Docker compose version (type docker-compose --version): Docker Compose version v2.24.5
System info (type uname -a): Linux orion 6.1.0-18-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.76-1 (2024-02-01) x86_64 GNU/Linux

Docker compose

version: "3.8"
services:
## DSMR-Reader
  dsmr-reader:
#    image: xirixiz/dsmr-reader-docker:5.10.4-2023.11.01
    image: xirixiz/dsmr-reader-docker:latest
    container_name: dsmr-reader
    restart: unless-stopped
    depends_on:
      - dsmr-db
    ports:
      - 7779:443
      - 7777:80
    cap_add:
      - NET_ADMIN
    volumes:
      - /data/docker-data/generic/etc/resolv.conf:/etc/resolv.conf
#      - /dev:/dev
    environment:
      - VACUUM_DB_ON_STARTUP=false
      - DSMRREADER_ADMIN_USER=admin
      - DSMRREADER_ADMIN_PASSWORD=<redacted>
      - VIRTUAL_HOST=dsmr.home.daxy.net
      - TZ=Europe/Amsterdam
      - DJANGO_DATABASE_HOST=192.168.1.10
      - DJANGO_DATABASE_NAME=dsmrreader
      - DJANGO_SECRET_KEY=<redacted>
      - DJANGO_DATABASE_USER=dsmrreader
      - DJANGO_DATABASE_PASSWORD=<redacted>
      - DJANGO_TIME_ZONE=Europe/Amsterdam
    devices:
      - /dev/serial/by-id/usb-FTDI_FT232R_USB_UART_A12YXQVU-if00-port0:/dev/ttyUSB0
    healthcheck:
      test:
        [
          "CMD",
          "curl",
          "-Lsf",
          "http://127.0.0.1/about",
          "-o",
          "/dev/null",
          "-w",
          "HTTP_%{http_code}"
        ]
      interval: 10s
      timeout: 5s
      retries: 10

## PostgreSQL
  dsmr-db:
    container_name: dsmr-db
    image: postgres:15.5
    restart: unless-stopped
    shm_size: "256m"
    ports:
      - 5432:5432
    volumes:
      - /data/docker-data/postgresql/data:/var/lib/postgresql/data
    environment:
      - POSTGRES_PASSWORD=<redacted>
      - POSTGRES_USER=dsmrreader
      - TZ=Europe/Amsterdam

Container logs

daxy@orion:/data/docker-data$ docker logs dsmr-reader s6-rc: info: service s6rc-oneshot-runner: starting s6-rc: info: service s6rc-oneshot-runner successfully started s6-rc: info: service fix-attrs: starting s6-rc: info: service docker-entrypoint: starting s6-rc: info: service fix-attrs successfully started s6-rc: info: service legacy-cont-init: starting [ INFO ] DSMR Reader Reader release: 5.11.0 [ INFO ] DSMR Reader Reader Docker release: 2024.02.03 s6-rc: info: service legacy-cont-init successfully started

───────────────────────────────────────────────────
 ___  ___ __  __ ___   ___             _
|   \/ __|  \/  | _ \ | _ \___ __ _ __| |___ _ _
| |) \__ \ |\/| |   / |   / -_) _` / _` / -_) '_|
|___/|___/_|  |_|_|_\ |_|_\___\__,_\__,_\___|_|
             __           _
            |   \ ___  __| |_____ _ _
            | |) / _ \/ _| / / -_) '_|
            |___/\___/\__|_\_\___|_|

DSMR-protocol reader, telegram data storage and
        energy consumption visualizer.

───────────────────────────────────────────────────
User UID: 803
User GID: 803
───────────────────────────────────────────────────

[ INFO ] Ensure permissions... [ INFO ] Setting architecture requirements... [ INFO ] X64 Architecture [ INFO ] Verifying if the DSMR Reader web credential variables have been set... [ INFO ] Verifying database connectivity to host: 192.168.1.10 with port: 5432... Connection to 192.168.1.10 5432 port [tcp/postgresql] succeeded! [ INFO ] Database connectivity successfully verified! [ INFO ] Running post configuration... Operations to perform: Apply all migrations: admin, auth, contenttypes, dsmr_api, dsmr_backend, dsmr_backup, dsmr_consumption, dsmr_datalogger, dsmr_dropbox, dsmr_frontend, dsmr_influxdb, dsmr_mindergas, dsmr_mqtt, dsmr_notification, dsmr_pvoutput, dsmr_stats, dsmr_weather, sessions Running migrations: No migrations to apply.

559 static files copied to '/var/www/dsmrreader/static'. Updating password of superuser "daxy" Deactivating any other existing superusers [ INFO ] Checking for CLIENT CERTIFICATE AUTHENTICATION configuration... [ INFO ] ENABLE_CLIENTCERT_AUTH is disabled, nothing to see here. Continuing... [ INFO ] Checking for NGINX SSL configuration... [ INFO ] ENABLE_NGINX_SSL is disabled, nothing to see here. Continuing... [ INFO ] Checking for HTTP AUTHENTICATION configuration... [ INFO ] ENABLE_HTTP_AUTH is disabled, nothing to see here. Continuing... [ INFO ] Configuring DSMR Reader to run all processes in a single container with a serial or ipv4 datalogger... [ INFO ] Using a serial connection for the DSMR Reader remote datalogger... [ INFO ] Fixing /dev/ttyUSB0 security... [ INFO ] Adding serial input method configuration settings... s6-rc: info: service docker-entrypoint successfully started s6-rc: info: service svc-dsmr-webinterface: starting s6-rc: info: service svc-dsmr-remote-datalogger: starting s6-rc: info: service svc-dsmr-datalogger: starting s6-rc: info: service svc-dsmr-backend: starting s6-rc: info: service svc-dsmr-webinterface successfully started s6-rc: info: service svc-dsmr-remote-datalogger successfully started s6-rc: info: service svc-dsmr-datalogger successfully started s6-rc: info: service svc-dsmr-backend successfully started s6-rc: info: service legacy-services: starting Starting DSMR Reader - webinterface... Starting DSMR Reader - datalogger... Starting DSMR Reader - backend... s6-rc: info: service legacy-services successfully started [2024-02-27 13:48:59 +0100] [95] [INFO] Starting gunicorn 20.1.0 [2024-02-27 13:48:59 +0100] [95] [INFO] Listening at: unix:/tmp/gunicorn--dsmr_webinterface.socket (95) [2024-02-27 13:48:59 +0100] [95] [INFO] Using worker: sync [2024-02-27 13:48:59 +0100] [124] [INFO] Booting worker with pid: 124 Current logging level set to "ERROR". More information can be found here: https://dsmr-reader.readthedocs.io/en/latest/how-to/troubleshooting/enabling-debug-logging.html Current logging level set to "ERROR". More information can be found here: https://dsmr-reader.readthedocs.io/en/latest/how-to/troubleshooting/enabling-debug-logging.html

Additional info

Connection is instantly disconnected:

daxy@orion:/data/docker-data$ telnet localhost 7777
Trying ::1...
Connected to localhost.
Escape character is '^]'.
Connection closed by foreign host.

FYI: The "healthcheck" section in my docker-compose file has just been added prior to submitting this ticket to ensure my file matches the recommended version. Removing it won't fix the issue.

Maximus48p commented 4 months ago

Seems that i have the same problem.

mrjmg commented 4 months ago

Same here. I reverted back to 5.10.4-2023.11.01. Datalogger did it's work in the background, seems to be isolated to just the webui not starting.

xirixiz commented 4 months ago

Switched to a newer component within the Docker image (S6 overlay). Currently I`m not using DSMR Reader Docker myself (due to Niko Home Control), so I was unable to validate upfront. Not sure how to solve this, so rolling back.

daxy01 commented 4 months ago

Thanks. Happy to test things and help out when needed if you'd like to test a newer version in the future, just say the word.

canedje commented 4 months ago

same problem over here. Giving a debug logging error:

Starting DSMR Reader - datalogger...
Starting DSMR Reader - webinterface...
Starting DSMR Reader - backend...
Current logging level set to "ERROR". More information can be found here: https://dsmr-reader.readthedocs.io/en/latest/how-to/troubleshooting/enabling-debug-logging.html
Current logging level set to "ERROR". More information can be found here: https://dsmr-reader.readthedocs.io/en/latest/how-to/troubleshooting/enabling-debug-logging.html

Going to that webpage it says to remove ### somewhere:

mrjmg commented 4 months ago

I tried debug logging, but that seems to be related to datalogger only. netstat -tulpn in the container lacks a service listening on port 80, it is in the previous one:

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      277/nginx -g daemon

Tried to capture those logs, but they're going to stderr/stdout (so I'd ecpect them in docker logs, don't see any nginx related errors/warnings there).

same problem over here. Giving a debug logging error:

Starting DSMR Reader - datalogger...
Starting DSMR Reader - webinterface...
Starting DSMR Reader - backend...
Current logging level set to "ERROR". More information can be found here: https://dsmr-reader.readthedocs.io/en/latest/how-to/troubleshooting/enabling-debug-logging.html
Current logging level set to "ERROR". More information can be found here: https://dsmr-reader.readthedocs.io/en/latest/how-to/troubleshooting/enabling-debug-logging.html

I get this same logging level messages in the previous version I'm now using (5.10.4-2023.11.01) - I don't think this is an error in itself.

xirixiz commented 4 months ago

Please read previous comments. I already shared why this is happening. Rolling back now.

https://github.com/xirixiz/dsmr-reader-docker/actions/runs/8065827764

daxy01 commented 4 months ago

I can confirm that the very latest version (from ~40m ago) works again. Thanks for your quick fix/rollback and support!

xirixiz / dsmr-reader-docker