[HELP] Not found config for custom modbus connectors

[javier-alvarez-ctic]

Describe the issue I'm currently working on deploying a plug & play instance of Thingsboard along with a Gateway and a client that will add the Gateway to the list of devices on Thingsboard. As of now, I have four custom Modbus connectors for the devices within my organization. However, I'm encountering errors during this process.

Configuration (Attach your configuration file) I am trying to deploy all of these services with this docker-compose.yml file:

version: '3.0'
services:
  thingsboard:
    restart: always
    image: "thingsboard/tb-postgres:${THINGSBOARD_VERSION:-3.6.2}" 
    ports:
      - "${THINGSBOARD_PORT:-8080}:9090" # HTTP port
      - "${MQTT_PORT}:1883" # MQTT Port
      - "${RPC_PORT}:7070" # Edge RPC Port
      - "${UDP_PORT_MIN}-${UDP_PORT_MAX}:5683-5688/udp" # UDP Port (to exposed internal COAP and LwM2m2 port)
    environment:
      TB_QUEUE_TYPE: ${TB_QUEUE_TYPE:-in-memory} # Can use an external queue, for example mqtt (need more environment config). In memory is not suitable for clusters.
      SSL_ENABLED: ${THINGSBOARD_USE_SSL:-false}
      MONITORING_ENABLED : ${MONITORING_ENABLED:-false}     
    volumes:
      - ${TB_DATA_FOLDER}:/data
      - ${TB_LOGS_FOLDER}:/var/log/thingsboard
    networks:
      - network_default        

  tb-gateway:
    restart: always
    image: "thingsboard/tb-gateway:${THINGSBOARD_GW_VERSION:-3.4.4}" #3.2
    # Ports (Required for REST connector)
    ports:
      - "${GATEWAY_PORT}:5000"
    environment:
      host: thingsboard
      port: 1883
      accessToken: ${TB_GW_CREDENTIAL_VALUE}      
      DEBUG: "True"
    volumes:
      - ${TB_GW_FOLDER}/config:/thingsboard_gateway/config
      - ${TB_GW_FOLDER}/logs:/thingsboard_gateway/logs
      - ${TB_GW_FOLDER}/extensions:/thingsboard_gateway/extensions
    depends_on:
      - thingsboard            
    networks:
      - network_default

  thingsboard-setup:
    build:
      context: ./client
      dockerfile: dockerfile
    environment:
      THINGSBOARD_HOST: ${THINGSBOARD_HOST:-thingsboard}
      THINGSBOARD_PORT: 9090
      THINGSBOARD_USE_SSL: ${THINGSBOARD_USE_SSL:-False}
      THINGSBOARD_USER: ${THINGSBOARD_USER}
      THINGSBOARD_PASSWORD: ${THINGSBOARD_PASS}
      THINGSBOARD_GW_NAME: ${THINGSBOARD_GW_NAME:-IoT_Gateway}
      TB_GW_CREDENTIAL_TYPE: ${TB_GW_CREDENTIAL_TYPE:-ACCESS_TOKEN}
      TB_GW_CREDENTIAL_VALUE: ${TB_GW_CREDENTIAL_VALUE}  
    depends_on:
      - thingsboard   
    deploy:
      restart_policy:
        condition: on-failure
        delay: 20s
        max_attempts: 15
        window: 60s
    networks:
      - network_default          

networks:
  network_default: 
    driver: bridge      

My folders are the following (I added the chown -R 799:799 permissons):

TB_DATA_FOLDER=~/.mytb-data
TB_LOGS_FOLDER=~/.mytb-logs
TB_GW_FOLDER=~/.tb-gateway

An my tb_gateway.json file is the following:

{
    "thingsboard": {
      "host": "thingsboard",
      "port": 1883,
      "remoteShell": false,
      "remoteConfiguration": false,
      "statistics": {
        "enable": false,
        "statsSendPeriodInSeconds": 3600,
        "configuration": "statistics.json"
      },
      "maxPayloadSizeBytes": 1024,
      "minPackSendDelayMS": 0,
      "checkConnectorsConfigurationInSeconds": 60,
      "handleDeviceRenaming": true,
      "checkingDeviceActivity": {
        "checkDeviceInactivity": false,
        "inactivityTimeoutSeconds": 120,
        "inactivityCheckPeriodSeconds": 10
      },
      "security": {
        "accessToken": "<TOKEN>"
      },
      "qos": 1
    },
    "storage": {
      "type": "memory",
      "read_records_count": 100,
      "max_records_count": 100000
    },
    "grpc": {
      "enabled": false,
      "serverPort": 9595,
      "keepaliveTimeMs": 10000,
      "keepaliveTimeoutMs": 5000,
      "keepalivePermitWithoutCalls": true,
      "maxPingsWithoutData": 0,
      "minTimeBetweenPingsMs": 10000,
      "minPingIntervalWithoutDataMs": 5000
    },
    "connectors": [
      {
        "name": "Modbus Connector",
        "type": "modbus",
        "configuration": "modbus_fancoil.json"
      },
      {
        "name": "Huawei inverter Connector",
        "type": "modbus",
        "configuration": "huawei_solar_inverter.json"
      },
      {
        "name": "Victron inverter Connector",
        "type": "modbus",
        "configuration": "victron_solar_inverter.json"
      },
      {
        "name": "Pylontech battery controller",
        "type": "modbus",
        "configuration": "pylontech_batteries.json"
      }
    ]
  }

The gateway is added to my Thingsboard instance successfully and I saw its Active status in the device list. If I connect to the "tb-gateway" container and go to the "/thingsboard_gateway/config" directory I see the files correctly.

Connector name (If you need help with some connector/converter): I have problems with custom ModbusTCP connector, I have three like the following one (I can access to its IPs addresses):

{
  "master": {
    "slaves": [
      {
        "host": "XXX.XXX.XXX.XXX",
        "port": XXX ,
        "type": "tcp",
        "method": "socket",
        "timeout": 35,
        "byteOrder": "BIG",
        "wordOrder": "BIG",
        "retries": true,
        "retryOnEmpty": true,
        "retryOnInvalid": true,
        "pollPeriod": 5000,
        "unitId": 1,
        "deviceName": " Inverter",
        "deviceType": "INVERTER",
        "sendDataOnlyOnChange": false,
        "connectAttemptTimeMs": 5000,
        "connectAttemptCount": 5,
        "waitAfterFailedAttemptsMs": 300000,
        "attributes": [],
        "timeseries": [
          {
            "address": 32016,
            "type": "16int",
            "tag": "mppt1_voltage",
            "objectsCount": 1,
            "functionCode": 3,
            "multiplier": 0.1
           },
           ...
          {
            "address": 32087,
            "type": "16int",
            "tag": "temperature",
            "objectsCount": 1,
            "functionCode": 3,
            "multiplier": 0.1
          }
        ],
          "attributeUpdates": [],
          "rpc": []
      }
    ]
  }
}

I copy my modbus connectors JSON files inside the folder tb-gateway:/thingsboard_gateway/config (inside the docker container).

Error traceback (If it was raised):

thingsboard-tb-gateway-1  | [notice] A new release of pip is available: 23.3.2 -> 24.0
thingsboard-tb-gateway-1  | [notice] To update, run: pip install --upgrade pip
thingsboard-tb-gateway-1  | Modbus library not found - installing...
thingsboard-tb-gateway-1  | 2024-02-12 12:28:09 - |INFO| - [tb_loader.py] - tb_loader - import_module - 68 - Import ModbusConnector from /thingsboard_gateway/connectors/modbus.
thingsboard-tb-gateway-1  | 2024-02-12 12:28:09 - |WARNING| - [tb_gateway_service.py] - tb_gateway_service - __connect_with_connectors - 804 - Config not found or empty for modbus
thingsboard-tb-gateway-1  | 2024-02-12 12:28:09 - |WARNING| - [tb_gateway_service.py] - tb_gateway_service - __connect_with_connectors - 804 - Config not found or empty for modbus
thingsboard-tb-gateway-1  | 2024-02-12 12:28:09 - |WARNING| - [tb_gateway_service.py] - tb_gateway_service - __connect_with_connectors - 804 - Config not found or empty for modbus
thingsboard-tb-gateway-1  | 2024-02-12 12:28:09 - |WARNING| - [tb_gateway_service.py] - tb_gateway_service - __connect_with_connectors - 804 - Config not found or empty for modbus
thingsboard-tb-gateway-1  | 2024-02-12 12:28:09 - |INFO| - [tb_gateway_service.py] - tb_gateway_service - __init__ - 304 - Gateway started.

Versions (please complete the following information):

• OS: Ubuntu 20.04
• Docker Thingsboard IoT Gateway image version 3.4.4
• Docker Thingsboard version 3.6.2

Thank all!

[github-actions[bot]]

Hi @javier-alvarez-ctic.
Thank you for your interest in ThingsBoard IoT Gateway.
Your issue was registered, please wait for response from engineer.

IOTGW-116

[javier-alvarez-ctic]

Additionally, I have another small question. I'm looking to enable hot reload in this deployment. To accomplish this, I've added the following line within the tb-gateway service in the Docker Compose configuration:

command: ["/bin/sh", "-c", "pip3 install -r requirements.txt && pip3 install pyserial-asyncio && python3 thingsboard_gateway/tb_gateway.py $TB_HOT_RELOAD"]

The value of TB_HOT_RELOAD is set to True. Could someone please confirm if this approach is correct?

Thank you!"

[javier-alvarez-ctic]

Hi again.

I tried to downgrade Thingsboard and tb-gateway versions and now its working:

version: '3.0'
services:
  thingsboard:
    restart: always
    image: "thingsboard/tb-postgres:3.4.1 

...

  tb-gateway:
    restart: always
    image: "thingsboard/tb-gateway:3.2"
    command: ["/bin/sh", "-c", "pip3 install -r requirements.txt && pip3 install pyserial-asyncio && pip install 'pymodbus<3.4.0' - -force-reinstall && python3 thingsboard_gateway/tb_gateway.py $TB_HOT_RELOAD"]

...

(But I need to fix the pymodbus version as I have seen in this source)

Thanks!

[imbeacon]

Hi @javier-alvarez-ctic,

Thank you for your interest in ThingsBoard IoT gateway, could you try to use a version from the master branch, it contains some changes in module that imports connectors and may help.

[fabik111]

Hi, I've had the same problem with the python 3 deb package, I downgraded to version 3.4.2 and worked for me.

[javier-alvarez-ctic]

Hi!

Yes, If i downgrade the gateway services or the pyrhon package its works. It seems to be a bug in the most recent(3.4.4) version of the docker image.

I am using an older version while it is fixed.

Thanks!!

[samson0v]

Hi @javier-alvarez-ctic, thanks for your interest in ThingsBoard IoT Gateway! In the latest version of the gateway, we abandoned mounts and instead used docker volumes, so you have two options to solve the problem:

1. copy your configuration files into the appropriate docker volume;
2. use remote configuration feature

Also, I attach you sample docker-compose file:

version: '3.4'
services:
  # ThingsBoard IoT Gateway Service Configuration
  tb-gateway:
    image: thingsboard/tb-gateway
    container_name: tb-gateway
    restart: always

    # Ports bindings - required by some connectors
    ports:
        - "5000:5000" # Comment if you don't use REST connector and change if you use another port
        # Uncomment and modify the following ports based on connector usage:
#        - "1052:1052" # BACnet connector
#        - "5026:5026" # Modbus TCP connector (Modbus Slave)
#        - "50000:50000/tcp" # Socket connector with type TCP
#        - "50000:50000/udp" # Socket connector with type UDP

    # Environment variables
    environment:
      - host=mqtt.thingsboard.cloud
      - port=1883
      - accessToken=YOUR_ACCESS_TOKEN

    # Volumes bind
    volumes:
      - tb-gw-config:/thingsboard_gateway/config
      - tb-gw-logs:/thingsboard_gateway/logs
      - tb-gw-extensions:/thingsboard_gateway/extensions

# Volumes declaration for configurations, extensions and configuration
volumes:
  tb-gw-config:
    name: tb-gw-config
  tb-gw-logs:
    name: tb-gw-logs
  tb-gw-extensions:
    name: tb-gw-extensions

[samson0v]

Hi @javier-alvarez-ctic, any updates?

[javier-alvarez-ctic]

Hi!!

Apologies for the delay in getting back to you; I had to jump to other projects. I am glad to inform you that with the proposed changes, everything is working correctly. I really appreciate all the help and effort on the Thingsboard project.

Thank you very much for everything!