search

xirixiz / dsmr-reader-docker

DSMR Reader in Docker.
https://hub.docker.com/r/xirixiz/dsmr-reader-docker
114 stars 33 forks source link

ENV for Remote datalogger is not used #303

Closed Ankh- closed 1 year ago

Ankh- commented 1 year ago

Support guidelines

I've found an issue and checked that ...

Description

Got a new docker installation running with the following snippet:

      - DSMRREADER_REMOTE_DATALOGGER_MODE=standalone
      - DSMRREADER_REMOTE_DATALOGGER_INPUT_METHOD=ipv4
      - DSMRREADER_REMOTE_DATALOGGER_NETWORK_HOST=192.168.30.10
      - DSMRREADER_REMOTE_DATALOGGER_NETWORK_PORT=23

I've got a P1 E20 ethernet (Zuidwijk) running nicely. When checking telnet for data, there is actual data. So something is wrong regarding the docker compose ENV.

Expected behaviour

Actual data!

Actual behaviour

Errors :(

dsmr | 2022-12-07 20:38:06,236 ERROR mixins run_once 107 | dsmr_datalogger.management.commands.dsmr_datalogger: [!] Exception raised. Traceback (most recent call last): dsmr | File "/usr/local/lib/python3.11/site-packages/serial/serialposix.py", line 322, in open dsmr | self.fd = os.open(self.portstr, os.O_RDWR | os.O_NOCTTY | os.O_NONBLOCK) dsmr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ dsmr | FileNotFoundError: [Errno 2] No such file or directory: '/dev/ttyUSB0' dsmr | dsmr | During handling of the above exception, another exception occurred: dsmr | dsmr | Traceback (most recent call last): dsmr | File "/app/dsmr_datalogger/scripts/dsmr_datalogger_api_client.py", line 39, in read_telegram dsmr | serial_handle = serial.serial_for_url( dsmr | ^^^^^^^^^^^^^^^^^^^^^^ dsmr | File "/usr/local/lib/python3.11/site-packages/serial/init.py", line 90, in serial_for_url dsmr | instance.open() dsmr | File "/usr/local/lib/python3.11/site-packages/serial/serialposix.py", line 325, in open dsmr | raise SerialException(msg.errno, "could not open port {}: {}".format(self._port, msg)) dsmr | serial.serialutil.SerialException: [Errno 2] could not open port /dev/ttyUSB0: [Errno 2] No such file or directory: '/dev/ttyUSB0' dsmr | dsmr | The above exception was the direct cause of the following exception: dsmr | dsmr | Traceback (most recent call last): dsmr | File "/app/dsmr_backend/mixins.py", line 96, in run_once dsmr | self.run(data=self.data, **options) dsmr | File "/app/dsmr_datalogger/management/commands/dsmr_datalogger.py", line 29, in run dsmr | telegram = next(self.telegram_generator) dsmr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ dsmr | File "/app/dsmr_datalogger/scripts/dsmr_datalogger_api_client.py", line 43, in read_telegram dsmr | raise RuntimeError("Failed to connect: {}", error) from error dsmr | RuntimeError: ('Failed to connect: {}', SerialException(2, "could not open port /dev/ttyUSB0: [Errno 2] No such file or directory: '/dev/ttyUSB0'")) dsmr |

Steps to reproduce

basic yaml with added remote datalogger

Docker info

root@box:/docker/dsmr# docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Docker Buildx (Docker Inc., v0.9.1-docker)
  compose: Docker Compose (Docker Inc., v2.13.0)
  scan: Docker Scan (Docker Inc., v0.21.0)

Server:
 Containers: 2
  Running: 0
  Paused: 0
  Stopped: 2
 Images: 3
 Server Version: 20.10.21
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 770bd0108c32f3fb5c73ae1264f7e503fe7b2661
 runc version: v1.1.4-0-g5fd4c4d
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.10.0-19-amd64
 Operating System: Debian GNU/Linux 11 (bullseye)
 OSType: linux
 Architecture: x86_64
 CPUs: 6
 Total Memory: 15.44GiB
 Name: box
 ID: BISR:P377:UHHH:HYO6:4JHT:XYT2:4JGN:XXEQ:OAKF:7WUK:VJ3T:XZAK
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Version

see docker info

Docker compose

root@box:/docker/dsmr# docker compose config
name: dsmr
services:
  dsmr:
    cap_add:
    - NET_ADMIN
    container_name: dsmr
    depends_on:
      dsmrdb:
        condition: service_healthy
    environment:
      DGID: "1000"
      DJANGO_TIME_ZONE: Europe/Amsterdam
      DSMRREADER_REMOTE_DATALOGGER_INPUT_METHOD: ipv4
      DSMRREADER_REMOTE_DATALOGGER_MODE: standalone
      DSMRREADER_REMOTE_DATALOGGER_NETWORK_HOST: 192.168.30.10
      DSMRREADER_REMOTE_DATALOGGER_NETWORK_PORT: "23"
      DUID: "1000"
      TZ: Europe/Amsterdam
      VIRTUAL_HOST: localhost
    healthcheck:
      test:
      - CMD
      - curl
      - -Lsf
      - http://127.0.0.1/about
      - -o
      - /dev/null
      - -w
      - HTTP_%{http_code}
      timeout: 5s
      interval: 10s
      retries: 10
    image: xirixiz/dsmr-reader-docker:latest
    links:
    - dsmrdb
    networks:
      default: null
    ports:
    - mode: ingress
      target: 80
      published: "7777"
      protocol: tcp
    - mode: ingress
      target: 443
      published: "7779"
      protocol: tcp
    restart: always
    volumes:
    - type: bind
      source: /etc/localtime
      target: /etc/localtime
      read_only: true
      bind:
        create_host_path: true
    - type: bind
      source: /docker/dsmr/dsmr_backups
      target: /app/backups
      bind:
        create_host_path: true
  dsmrdb:
    container_name: dsmrdb
    environment:
      DGID: "1000"
      DUID: "1000"
      PG_TZ: Europe/Amsterdam
      POSTGRES_DB: dsmrreader
      POSTGRES_PASSWORD: dsmrreader
      POSTGRES_USER: dsmrreader
      TZ: Europe/Amsterdam
    healthcheck:
      test:
      - CMD-SHELL
      - pg_isready -U dsmrreader
      timeout: 5s
      interval: 10s
      retries: 10
    image: postgres:14-alpine
    networks:
      default: null
    restart: always
    volumes:
    - type: bind
      source: /docker/dsmr/dsmrdb
      target: /var/lib/postgresql/data
      bind:
        create_host_path: true
networks:
  default:
    name: dsmr_default
root@box:/docker/dsmr#

Container logs

see errors

Additional info

No response

xirixiz commented 1 year ago

Hi, it seems DSMRREADER_REMOTE_DATALOGGER_MODE is not set correctly. This should be set to "receiver"

https://github.com/xirixiz/dsmr-reader-docker#dsmr-datalogger-related

Can you try and validate this? Thanks!

Ankh- commented 1 year ago

Hi. Changed to receiver and removed all data (clean run)

root@box:/docker/dsmr# cat docker-compose.yaml | grep DATALOGGER

  • DSMRREADER_REMOTE_DATALOGGER_MODE=receiver
  • DSMRREADER_REMOTE_DATALOGGER_INPUT_METHOD=ipv4
  • DSMRREADER_REMOTE_DATALOGGER_NETWORK_HOST=192.168.30.10
  • DSMRREADER_REMOTE_DATALOGGER_NETWORK_PORT=23 root@box:/docker/dsmr# ls -al total 12 drwxr-xr-x 2 paul paul 4096 Dec 7 21:03 . drwxr-xr-x 5 root root 4096 Dec 7 15:53 .. -rw-r--r-- 1 paul paul 1911 Dec 7 21:03 docker-compose.yaml root@box:/docker/dsmr#

The errors are gone. But when checking the dsmr page it still gives no data. Or do I need to go the admin level and setup the datalogger settings there? I thought it would be forced, so after a docker reboot it isn't back to defaults.

xirixiz commented 1 year ago

Hi, maybe @zuidwijk can help himself. He's been active here as well. Maybe it requires a different set of configuration. I`m not sure.

Ankh- commented 1 year ago

Dont think the issue is with the P1 ethernet reader from zuidwijk.

Just switched back to standalone and manually setup the settings: http://x.x.x.x:7777/admin/dsmr_datalogger/dataloggersettings/

And now it works. Yet when stopping the compose and starting it will wipe this setting. So something is going wrong with the environment in the docker-compose.yml file.

This is the current snippet:

dsmr: image: xirixiz/dsmr-reader-docker:latest depends_on: dsmrdb: condition: service_healthy container_name: dsmr links:

  • dsmrdb cap_add:
  • NET_ADMIN restart: always volumes:
  • /etc/localtime:/etc/localtime:ro
  • /home/dsmr/dsmr_backups:/app/backups environment:
  • TZ=Europe/Amsterdam
  • DJANGO_TIME_ZONE=Europe/Amsterdam
  • VIRTUAL_HOST=localhost
  • DSMRREADER_REMOTE_DATALOGGER_MODE=standalone
  • DSMRREADER_REMOTE_DATALOGGER_INPUT_METHOD=ipv4
  • DSMRREADER_REMOTE_DATALOGGER_NETWORK_HOST=192.168.30.10
  • DSMRREADER_REMOTE_DATALOGGER_NETWORK_PORT=23
  • DSMRREADER_REMOTE_DATALOGGER_DEBUG_LOGGING=true
  • DUID=1000
  • DGID=1000 ports:
  • 7777:80
  • 7779:443

    healthcheck:

    disable: true

    healthcheck: test: [ "CMD", "curl", "-Lsf", "http://127.0.0.1/about", "-o", "/dev/null", "-w", "HTTP_%{http_code}" ] interval: 10s timeout: 5s retries: 10

I've setup the debug log for the datalogger aswell and it doesnt show any debug log....

xirixiz commented 1 year ago

What I meant is that Zuidwijk knows exactly how to set it up properly. I'm sure Zuidwijks solution is working fine. What is it you change after starting the container?

Ankh- commented 1 year ago

Ik switch even naar nederlands ;-)

De P1 ethernet verzorgt alleen maar een telegram/console. Dus de DSMR Reader moet daar z'n info vandaan halen. Ik heb het idee als ik de remote_datalogger_mode als receiver in stel, deze niet meer externe informatie gaat opvragen. En enkel als ontvanger (via de API) z'n gegevens kan ontvangen.

Momenteel heb ik de remote_datalogger_mode als standalone ingesteld en middels de admin pagina wel de P1 Ethernet reader ingesteld. En nu werkt het allemaal.

zuidwijk commented 1 year ago

Het ziet er allemaal ingewikkelder uit dan dat hoe ik het draai. Dit is mijn docker compose:



version: '3.8'

services:
  dsmrdb:
    image: postgres:12.4-alpine
    container_name: dsmrdb
    restart: always
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ./dsmrdb:/var/lib/postgresql/data
    environment:
      - TZ=Europe/Amsterdam
      - PG_TZ=Europe/Amsterdam
      - POSTGRES_USER=dsmrreader
      - POSTGRES_PASSWORD=dsmrreader
      - POSTGRES_DB=dsmrreader
  dsmr:
    image: xirixiz/dsmr-reader-docker:latest
    container_name: dsmr
    depends_on:
      - dsmrdb
    cap_add:
      - NET_ADMIN    
    links:
      - dsmrdb
    restart: always
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ./dsmr_backups:/dsmr/backups
    environment:
      - TZ=Europe/Amsterdam
      - DJANGO_TIME_ZONE=Europe/Amsterdam
      - VIRTUAL_HOST=localhost
      - DSMRREADER_ADMIN_USER=****
      - DSMRREADER_ADMIN_PASSWORD=****
    ports:
      - 7777:80
      - 7779:443
zuidwijk commented 1 year ago

Ik switch even naar nederlands ;-)

De P1 ethernet verzorgt alleen maar een telegram/console. Dus de DSMR Reader moet daar z'n info vandaan halen. Ik heb het idee als ik de remote_datalogger_mode als receiver in stel, deze niet meer externe informatie gaat opvragen. En enkel als ontvanger (via de API) z'n gegevens kan ontvangen.

Momenteel heb ik de remote_datalogger_mode als standalone ingesteld en middels de admin pagina wel de P1 Ethernet reader ingesteld. En nu werkt het allemaal.

Ik stel het ip adres van de ethernet module in op de config pagina van dsmr-reader. image

xirixiz commented 1 year ago

Bedankt voor het delen 👍🏻. Ik zal de check aanpassen voor standalone. Ik merk dat het vaak verwarrend is wanneer /dev/ttyUSB niet gevonden wordt.

xirixiz commented 1 year ago

Readme updated.