thingsboard / thingsboard

Open-source IoT Platform - Device management, data collection, processing and visualization.
https://thingsboard.io
Apache License 2.0
16.95k stars 5.05k forks source link

Gateway connectors not visible in frontend when added via docker volumes #11003

Closed javier-alvarez-ctic closed 2 months ago

javier-alvarez-ctic commented 2 months ago

Describe the bug After deploying ThingsBoard with docker-compose and setting up a plug & play proof of concept using scripts and makefiles, the Modbus connectors are not visible in the UI under Paneles > ThingsBoard IoT Gateways > IoT Gateway > Connectors configuration, despite being present in the Docker volume. However, if the connectors are added manually via the UI, they appear correctly.

Your Server Environment

Docker compose file:

version: '3.0'
services:
#########################
###### THINGSBOARD ######
#########################
  thingsboard:
    restart: always
    image: "thingsboard/tb-postgres:${THINGSBOARD_VERSION:-3.6.2}" # 3.4.1 Also can use thingsboard/tb-cassandra (need minimum 4GB of RAM, but is recommended 8GB)
    ports:
      - "${THINGSBOARD_PORT:-8080}:9090" # HTTP port
      - "${MQTT_PORT}:1883" # MQTT Port
      - "${RPC_PORT}:7070" # Edge RPC Port
      - "${UDP_PORT_MIN}-${UDP_PORT_MAX}:5683-5688/udp" # UDP Port (to exposed internal COAP and LwM2m2 port)    
    environment:
      TB_QUEUE_TYPE: ${TB_QUEUE_TYPE:-in-memory} # Can use an external queue, for example mqtt (need more environment config). In memory is not suitable for clusters.
      SSL_ENABLED: ${THINGSBOARD_USE_SSL:-false}
      MONITORING_ENABLED : ${MONITORING_ENABLED:-false}
      DATABASE: ${DATABASE:-postgres}
      DATABASE_TS_TYPE: ${TB_DATABASE_TS_TYPE:-sql} # sql, cassandra or timescale
      DATABASE_TS_LATEST_TYPE: ${TB_DATABASE_TS_LATEST_TYPE:-sql} # sql, cassandra or timescale       
    volumes:
      - tb-data:/data
      - tb-logs:/var/log/thingsboard
    networks:
      - network_default        

  thingsboard-setup:
    build:
      context: ./client
      dockerfile: dockerfile
    environment:
      THINGSBOARD_HOST: ${THINGSBOARD_HOST:-thingsboard}
      THINGSBOARD_PORT: 9090
      THINGSBOARD_USE_SSL: ${THINGSBOARD_USE_SSL:-False}
      THINGSBOARD_USER: ${THINGSBOARD_USER}
      THINGSBOARD_PASSWORD: ${THINGSBOARD_PASS}
      THINGSBOARD_GW_NAME: ${THINGSBOARD_GW_NAME:-IoT_Gateway}
      TB_GW_CREDENTIAL_TYPE: ${TB_GW_CREDENTIAL_TYPE:-ACCESS_TOKEN}
      TB_GW_CREDENTIAL_VALUE: ${TB_GW_CREDENTIAL_VALUE}  
    depends_on:
      - thingsboard   
    deploy:
      restart_policy:
        condition: on-failure
        delay: 20s
        max_attempts: 20
        window: 60s
    networks:
      - network_default   

#########################
###### GatewayIoT ######
#########################
  tb-gateway:
    image: thingsboard/tb-gateway:${THINGSBOARD_GW_VERSION:-3.4.5}
    container_name: tb-gateway
    restart: always
    #command: ["/bin/sh", "-c", "pip3 install -r requirements.txt && pip3 install pyserial-asyncio && pip install 'pymodbus<3.4.0' --force-reinstall && python3 thingsboard_gateway/tb_gateway.py $TB_HOT_RELOAD"]
    # Ports bindings - required by some connectors
    ports:
        - "${GATEWAY_PORT}:5000" # Comment if you don't use REST connector and change if you use another port
        # Uncomment and modify the following ports based on connector usage:
#        - "1052:1052" # BACnet connector
#        - "5026:5026" # Modbus TCP connector (Modbus Slave)
#        - "50000:50000/tcp" # Socket connector with type TCP
#        - "50000:50000/udp" # Socket connector with type UDP

    # Environment variables
    environment:
      host: thingsboard
      port: 1883
      accessToken: ${TB_GW_CREDENTIAL_VALUE}      
      DEBUG: "True"

    # Volumes bind
    volumes:
      - tb-gw-config:/thingsboard_gateway/config
      - tb-gw-logs:/thingsboard_gateway/logs
      - tb-gw-extensions:/thingsboard_gateway/extensions
    depends_on: 
      - thingsboard-setup
    networks:
      - network_default      

#########################
###### Volumes ######
#########################
volumes:
  tb-data:
    name: tb-data
  tb-logs:
    name: tb-logs  
  tb-gw-config:
    name: tb-gw-config
  tb-gw-logs:
    name: tb-gw-logs
  tb-gw-extensions:
    name: tb-gw-extensions    

networks:
  network_default: 
    driver: bridge      

Gateway configuration:

{
    "thingsboard": {
      "host": "thingsboard",
      "port": 1883,
      "remoteShell": false,
      "remoteConfiguration": false,
      "statistics": {
        "enable": false,
        "statsSendPeriodInSeconds": 3600,
        "configuration": "statistics.json"
      },
      "maxPayloadSizeBytes": 1024,
      "minPackSendDelayMS": 0,
      "checkConnectorsConfigurationInSeconds": 60,
      "handleDeviceRenaming": true,
      "checkingDeviceActivity": {
        "checkDeviceInactivity": false,
        "inactivityTimeoutSeconds": 120,
        "inactivityCheckPeriodSeconds": 10
      },
      "security": {
        "accessToken": "XXXXXXXXXXXXXXXXXXXXXXXXXXXX"
      },
      "qos": 1
    },
    "storage": {
      "type": "memory",
      "read_records_count": 100,
      "max_records_count": 100000
    },
    "grpc": {
      "enabled": false,
      "serverPort": 9595,
      "keepaliveTimeMs": 10000,
      "keepaliveTimeoutMs": 5000,
      "keepalivePermitWithoutCalls": true,
      "maxPingsWithoutData": 0,
      "minTimeBetweenPingsMs": 10000,
      "minPingIntervalWithoutDataMs": 5000
    },
    "connectors": [
      {
        "name": "Simulator MQTT controller",
        "type": "mqtt",
        "configuration": "simulator_mqtt.json"
      },
      {
        "name": "Huawei inverter Connector",
        "type": "modbus",
        "configuration": "huawei_solar_inverter.json"
      },
      {
        "name": "Victron inverter Connector",
        "type": "modbus",
        "configuration": "victron_solar_inverter.json"
      },
      {
        "name": "Pylontech battery controller",
        "type": "modbus",
        "configuration": "pylontech_batteries.json"
      }
    ]
  }

Your Client Environment **Desktop

To Reproduce Steps to reproduce the behavior:

  1. Deploy ThingsBoard and Gateway using docker-compose.
  2. Load the Modbus connector configurations into the Docker volume (using script).
  3. Register the gateway on the ThingsBoard platform (using client).
  4. Navigate to Paneles > ThingsBoard IoT Gateways > IoT Gateway > Connectors configuration.
  5. Observe that the connectors are not visible in the UI, but it works.

Expected behavior The Modbus connectors added via scripts should be visible in the UI under the Connectors configuration, just as they are when added manually through the UI.

Screenshots image image

Additional context Service script for registering the gateway:

def main():
    # Creating the REST client object with context manager to get auto token refresh
    with RestClientCE(base_url=url) as rest_client:
        try:
            # Auth with credentials
            logging.info(f"usermane={TB_USERNAME} | password={TB_PASSWORD}")
            rest_client.login(username=TB_USERNAME, password=TB_PASSWORD)

            # Creating a Device
            default_device_profile_id = rest_client.get_default_device_profile_info().id
            device = Device(name=THINGSBOARD_GW_NAME, device_profile_id=default_device_profile_id, additional_info={"gateway": True, "overwriteActivityTime": False, "description": ""})
            credentials = DeviceCredentials(credentials_type=TB_GW_CREDENTIAL_TYPE, credentials_id=TB_GW_CREDENTIAL_VALUE)
            create_device_request = SaveDeviceWithCredentialsRequest(device=device, credentials=credentials)
            device = rest_client.save_device_with_credentials(create_device_request)

            logging.info(" Device was created:\n%r\n", device)
        except ApiException as e:
            logging.exception(e)
mykhailokornieiev commented 2 months ago

Hi. Your request is related to IoT Gateway, so please create a new ticket in the IoT Gateway issues https://github.com/thingsboard/thingsboard-gateway/issues