How add MQTT in time series tutorial?

[sfl0r3nz05]

Based on the IoT-over-MQTT tutorial, I have enabled MQTT instead of HTTP for the communication between sensor and agent. So far I have not been able to find the error in my configuration?

This is the configuration for the IoT Agent:

  # IoT-Agent JSON
  iot-agent:
    image: fiware/iotagent-ul:${ULTRALIGHT_VERSION}
    hostname: iot-agent
    container_name: fiware-iot-agent
    restart: always
    depends_on:
      - mongo-db
      - mosquitto
    networks:
      - default
    expose:
      - "${IOTA_NORTH_PORT}"
    ports:
      - "${IOTA_NORTH_PORT}:${IOTA_NORTH_PORT}" # localhost:4041
    environment:
      - IOTA_CB_HOST=orion # name of the context broker to update context
      - IOTA_CB_PORT=${ORION_PORT} # port the context broker listens on to update context
      - IOTA_NORTH_PORT=${IOTA_NORTH_PORT}
      - IOTA_REGISTRY_TYPE=mongodb #Whether to hold IoT device info in memory or in a database
      - IOTA_LOG_LEVEL=DEBUG # The log level of the IoT Agent
      - IOTA_TIMESTAMP=true # Supply timestamp information with each measurement
      - IOTA_CB_NGSI_VERSION=v2 # use NGSIv2 when sending updates for active attributes
      - IOTA_AUTOCAST=true # Ensure Ultralight number values are read as numbers not strings
      - IOTA_MONGO_HOST=mongo-db # The host name of MongoDB
      - IOTA_MONGO_PORT=${MONGO_DB_PORT} # The port mongoDB is listening on
      - IOTA_MONGO_DB=iotagentul # The name of the database used in mongoDB
      - IOTA_MQTT_HOST=mosquitto # The host name of the MQTT Broker
      - IOTA_MQTT_PORT=1883 # The port the MQTT Broker is listening on to receive topics
      - IOTA_DEFAULT_RESOURCE= # Default is blank. I'm using MQTT so I don't need a resource
      - IOTA_PROVIDER_URL=http://iot-agent:${IOTA_NORTH_PORT}
      - IOTA_DEFAULT_TRANSPORT=MQTT
    healthcheck:
      interval: 5s

This is the configuration for the IoT Sensor:

# Tutorial acts as a series of dummy IoT Sensors over HTTP
  iot-sensors:
    image: fiware/tutorials.context-provider
    hostname: iot-sensors
    container_name: iot-sensors
    restart: always
    depends_on:
      - orion
      - quantumleap
    expose:
      - "${TUTORIAL_APP_PORT}"
      - "${TUTORIAL_DUMMY_DEVICE_PORT}"
    ports:
      - "${TUTORIAL_APP_PORT}:${TUTORIAL_APP_PORT}" # localhost:3000
      - "${TUTORIAL_DUMMY_DEVICE_PORT}:${TUTORIAL_DUMMY_DEVICE_PORT}" # localhost:3001
    environment:
      - "MONGO_URL=mongodb://mongo-db:27017"
      - "DEBUG=iot-sensors:*"
      - "WEB_APP_PORT=${TUTORIAL_APP_PORT}" # Port used by the content provider proxy and web-app for viewing data
      - "IOTA_HTTP_HOST=iot-agent"
      - "IOTA_HTTP_PORT=${IOTA_SOUTH_PORT}"
      - "IOTA_DEFAULT_RESOURCE=/iot/d"
      - "MQTT_TOPIC_PROTOCOL=ul"
      - "DUMMY_DEVICES_PORT=${TUTORIAL_DUMMY_DEVICE_PORT}" # Port used by the dummy IOT devices to receive commands
      - "DUMMY_DEVICES_TRANSPORT=MQTT" # Default transport used by dummy IoT devices
      - "CONTEXT_BROKER=http://orion:${ORION_PORT}/v2" # URL of the context broker to update context
      - "NGSI_LD_PREFIX="
      - "CRATE_DB_SERVICE_URL=http://crate-db:4200/_sql"
      - "DUMMY_DEVICES_API_KEY=4jggokgpepnvsb2uv4s40d59ov"
      - "MQTT_BROKER_URL=mqtt://mosquitto"
    networks:
      - default

This is the configuration for the MQTT Broker:

  # MQTT broker
  mosquitto:
    image: eclipse-mosquitto:1.6.14
    hostname: mosquitto
    container_name: mosquitto
    restart: always
    expose:
      - "1883"
      - "9001"
    ports:
      - "1883:1883"
      - "9001:9001"
    volumes:
      - ./mosquitto/mosquitto.conf:/mosquitto/config/mosquitto.conf
    networks:
      - default

Following the time series tutorial recommendation I have tried to unlock the door:

... and this is the log obtained in the IoT Sensor with MQTT:

... this is the log obtained for the IoT Sensor without MQTT:

... this is the log obtained in the IoT Agent with MQTT:

... this is the log obtained in the Orion with MQTT:

[jason-fox]

It is probably because the dummy devices are still listening on HTTP and are not subscribing to MQTT Topics. Just set DUMMY_DEVICES_TRANSPORT=MQTT to switch the transport as shown here

[jason-fox]

Alternatively it could be to do with the provisioning of the device - make sure you have the MQTT transport like here

[sfl0r3nz05]

Thanks !!!

I got this message when I provisioned the sensor:

{"name":"DUPLICATE_DEVICE_ID","message":"A device with the same pair (Service, DeviceId) was found:motion001"}

I guess it will be because I have provided this configuration as part of the provision-devices file:

#!/bin/bash
#
#  curl commands to reload the data from the previous tutorial
#
#

set -e

printf "⏳ Provisioning IoT devices "

#
# Create a service groups for all UltraLight IoT devices
#

curl -s -o /dev/null -X POST \
  "http://iot-agent:$IOTA_NORTH_PORT/iot/services" \
  -H 'Content-Type: application/json' \
  -H 'fiware-service: openiot' \
  -H 'fiware-servicepath: /' \
  -d '{
 "services": [
   {
     "apikey":      "4jggokgpepnvsb2uv4s40d59ov",
     "cbroker":     "'"http://orion:$ORION_PORT"'",
     "entity_type": "Motion",
     "resource":    "/iot/d",
     "protocol":    "PDI-IoTA-UltraLight",
     "transport":   "MQTT",
     "timezone":    "Europe/Berlin",
     "attributes": [
       { "object_id": "c", "name":"count", "type":"Integer"}
      ],
      "static_attributes": [
          {"name": "category", "type":"Text", "value": ["sensor"]},
          {"name": "controlledProperty", "type": "Text", "value": "motion"},
          {"name": "function", "type": "Text", "value":["sensing"]},
          {"name": "supportedProtocol", "type": "Text", "value": ["ul20"]},
          {"name": "supportedUnits", "type": "Text", "value": "C62"}
      ]
   },
   {
     "apikey":      "4jggokgpepnvsb2uv4s40d59ov",
     "cbroker":     "'"http://orion:$ORION_PORT"'",
     "entity_type": "Bell",
     "resource":    "/iot/d",
     "protocol":    "PDI-IoTA-UltraLight",
     "transport":   "MQTT",
     "timezone":    "Europe/Berlin",
     "commands": [ 
        {
          "name": "ring",
          "type": "command"
        }
      ],
      "static_attributes": [
          {"name": "category", "type":"Text", "value": ["actuator"]},
          {"name": "controlledProperty", "type": "Text", "value": "noiseLevel"},
          {"name": "function", "type": "Text", "value":["onOff"]},
          {"name": "supportedProtocol", "type": "Text", "value": ["ul20"]}
      ]
   },
   {
     "apikey":      "4jggokgpepnvsb2uv4s40d59ov",
     "cbroker":     "'"http://orion:$ORION_PORT"'",
     "entity_type": "Lamp",
     "resource":    "/iot/d",
     "protocol":    "PDI-IoTA-UltraLight",
     "transport":   "MQTT",
     "timezone":    "Europe/Berlin",
     "commands": [ 
        {"name": "on","type": "command"},
        {"name": "off","type": "command"}
     ],
     "attributes": [
        {"object_id": "s", "name": "state", "type":"Text"},
        {"object_id": "l", "name": "luminosity", "type":"Integer",
          "metadata": {"unitCode": {"type": "Text","value": "CDL"}}
        }
     ],
     "static_attributes": [
          {"name": "category", "type":"Text", "value": ["actuator","sensor"]},
          {"name": "controlledProperty", "type": "Text", "value": "light"},
          {"name": "function", "type": "Text", "value":["onOff", "sensing"]},
          {"name": "supportedProtocol", "type": "Text", "value": ["ul20"]},
          {"name": "supportedUnits", "type": "Text", "value": "CDL"}
     ]
   },
   {
     "apikey":      "4jggokgpepnvsb2uv4s40d59ov",
     "cbroker":     "'"http://orion:$ORION_PORT"'",
     "entity_type": "Door",
     "resource":    "/iot/d",
     "protocol":    "PDI-IoTA-UltraLight",
     "transport":   "MQTT",
     "timezone":    "Europe/Berlin",
     "commands": [ 
        {"name": "unlock","type": "command"},
        {"name": "open","type": "command"},
        {"name": "close","type": "command"},
        {"name": "lock","type": "command"}
      ],
     "attributes": [
        {"object_id": "s", "name": "state", "type":"Text"}
      ],
     "static_attributes": [
          {"name": "category", "type":"Text", "value": ["actuator", "sensor"]},
          {"name": "controlledProperty", "type": "Text", "value": "state"},
          {"name": "function", "type": "Text", "value":["openClose", "eventNotification"]},
          {"name": "supportedProtocol", "type": "Text", "value": ["ul20"]}
      ]
   }
 ]
}'

####################################################
#
# Provision sensors for Store 001
#

curl -s -o /dev/null -X POST \
  "http://iot-agent:$IOTA_NORTH_PORT/iot/devices" \
  -H 'Content-Type: application/json' \
  -H 'fiware-service: openiot' \
  -H 'fiware-servicepath: /' \
  -d '{
 "devices": [
   {
     "device_id":   "motion001",
     "entity_name": "Motion:001",
     "entity_type": "Motion",
     "static_attributes": [
         {"name": "refStore", "type": "Relationship","value": "urn:ngsi-ld:Store:001"}
     ]
   },
   {
      "device_id": "bell001",
      "entity_name": "Bell:001",
      "entity_type": "Bell",
      "endpoint": "'"http://iot-sensors:$TUTORIAL_DUMMY_DEVICE_PORT/iot/bell001"'",
      "static_attributes": [
          {"name": "refStore", "type": "Relationship","value": "urn:ngsi-ld:Store:001"}
      ]
    },
    {
      "device_id": "door001",
      "entity_name": "Door:001",
      "entity_type": "Door",
      "endpoint": "'"http://iot-sensors:$TUTORIAL_DUMMY_DEVICE_PORT/iot/door001"'",
      "static_attributes": [
          {"name": "refStore", "type": "Relationship","value": "urn:ngsi-ld:Store:001"}
       ]
    }, 
    {
      "device_id": "lamp001",
      "entity_name": "Lamp:001",
      "entity_type": "Lamp",
      "endpoint": "'"http://iot-sensors:$TUTORIAL_DUMMY_DEVICE_PORT/iot/lamp001"'",
      "static_attributes": [
        {"name": "refStore", "type": "Relationship","value": "urn:ngsi-ld:Store:001"},
        {"name":"location", "type":"geo:json", "value": {
            "coordinates": [52.5547,13.3986],
            "type": "Point"
          }
        }
      ]
    }
  ]
}
'

####################################################
#
#
# Provision sensors for Store 002
#

curl -s -o /dev/null -X POST \
  "http://iot-agent:$IOTA_NORTH_PORT/iot/devices" \
  -H 'Content-Type: application/json' \
  -H 'fiware-service: openiot' \
  -H 'fiware-servicepath: /' \
  -d '{
 "devices": [
   {
     "device_id":   "motion002",
     "entity_name": "Motion:002",
     "entity_type": "Motion",
     "static_attributes": [
        {"name": "refStore", "type": "Relationship","value": "urn:ngsi-ld:Store:002"}
     ]
   },
   {
      "device_id": "bell002",
      "entity_name": "Bell:002",
      "entity_type": "Bell",
      "endpoint": "'"http://iot-sensors:$TUTORIAL_DUMMY_DEVICE_PORT/iot/bell002"'",
      "static_attributes": [
        {"name": "refStore", "type": "Relationship","value": "urn:ngsi-ld:Store:002"}
      ]
    },
    {
      "device_id": "door002",
      "entity_name": "Door:002",
      "entity_type": "Door",
      "endpoint": "'"http://iot-sensors:$TUTORIAL_DUMMY_DEVICE_PORT/iot/door002"'",
      "static_attributes": [
        {"name": "refStore", "type": "Relationship","value": "urn:ngsi-ld:Store:002"}
      ]
    },
    {
      "device_id": "lamp002",
      "entity_name": "Lamp:002",
      "entity_type": "Lamp",
      "endpoint": "'"http://iot-sensors:$TUTORIAL_DUMMY_DEVICE_PORT/iot/lamp002"'",
      "static_attributes": [
        {"name": "refStore", "type": "Relationship","value": "urn:ngsi-ld:Store:002"},
        {"name":"location", "type":"geo:json", "value": {
          "coordinates": [52.5075,13.3903],
          "type": "Point"}
        }
      ]
    }
  ]
}
'

####################################################
#
#
# Provision sensors for Store 3
#

curl -s -o /dev/null -X POST \
  "http://iot-agent:$IOTA_NORTH_PORT/iot/devices" \
  -H 'Content-Type: application/json' \
  -H 'fiware-service: openiot' \
  -H 'fiware-servicepath: /' \
  -d '{
 "devices": [
   {
     "device_id":   "motion003",
     "entity_name": "Motion:003",
     "entity_type": "Motion",
     "static_attributes": [
        {"name": "refStore", "type": "Relationship","value": "urn:ngsi-ld:Store:003"}
     ]
   },
   {
      "device_id": "bell003",
      "entity_name": "Bell:003",
      "entity_type": "Bell",
      "endpoint": "'"http://iot-sensors:$TUTORIAL_DUMMY_DEVICE_PORT/iot/bell003"'",
      "static_attributes": [
        {"name": "refStore", "type": "Relationship","value": "urn:ngsi-ld:Store:003"}
      ]
    },
    {
      "device_id": "door003",
      "entity_name": "Door:003",
      "entity_type": "Door",
      "endpoint": "'"http://iot-sensors:$TUTORIAL_DUMMY_DEVICE_PORT/iot/door003"'",
      "static_attributes": [
        {"name": "refStore", "type": "Relationship","value": "urn:ngsi-ld:Store:003"}
      ]
    },
    {
      "device_id": "lamp003",
      "entity_name": "Lamp:003",
      "entity_type": "Lamp",
      "endpoint": "'"http://iot-sensors:$TUTORIAL_DUMMY_DEVICE_PORT/iot/lamp003"'",
      "static_attributes": [
        {"name": "refStore", "type": "Relationship","value": "urn:ngsi-ld:Store:003"},
        {"name":"location", "type":"geo:json", "value": {
          "coordinates": [52.5031,13.4447],
          "type": "Point"}
        }
      ]
    }
  ]
}
'

####################################################
#
#
# Provision sensors for Store 4
#

curl -s -o /dev/null -X POST \
  "http://iot-agent:$IOTA_NORTH_PORT/iot/devices" \
  -H 'Content-Type: application/json' \
  -H 'fiware-service: openiot' \
  -H 'fiware-servicepath: /' \
  -d '{
 "devices": [
   {
     "device_id":   "motion004",
     "entity_name": "Motion:004",
     "entity_type": "Motion",
     "static_attributes": [
        {"name": "refStore", "type": "Relationship","value": "urn:ngsi-ld:Store:004"}
     ]
   },
   {
      "device_id": "bell004",
      "entity_name": "Bell:004",
      "entity_type": "Bell",
      "endpoint": "'"http://iot-sensors:$TUTORIAL_DUMMY_DEVICE_PORT/iot/bell004"'",
      "static_attributes": [
        {"name": "refStore", "type": "Relationship","value": "urn:ngsi-ld:Store:004"}
      ]
    },
    {
      "device_id": "door004",
      "entity_name": "Door:004",
      "entity_type": "Door",
      "endpoint": "'"http://iot-sensors:$TUTORIAL_DUMMY_DEVICE_PORT/iot/door004"'",
      "static_attributes": [
        {"name": "refStore", "type": "Relationship","value": "urn:ngsi-ld:Store:004"}
      ]
    },
    {
      "device_id": "lamp004",
      "entity_name": "Lamp:004",
      "entity_type": "Lamp",
      "endpoint": "'"http://iot-sensors:$TUTORIAL_DUMMY_DEVICE_PORT/iot/lamp004"'",
      "static_attributes": [
        {"name": "refStore", "type": "Relationship","value": "urn:ngsi-ld:Store:004"},
        {"name":"location", "type":"geo:json", "value": {
          "coordinates": [52.5208,13.4094],
          "type": "Point"}
        }
      ]
    }
  ]
}
'

echo -e " \033[1;32mdone\033[0m"

[jason-fox]

resource should be left blank for MQTT - see here

[sfl0r3nz05]

Thanks. Still troubleshooting ...

I have skipped the configuration of the provision-devices file to provide the provisioning recommended:

First, the provisioning of services:

curl -iX POST \
  'http://localhost:4041/iot/services' \
  -H 'Content-Type: application/json' \
  -H 'fiware-service: openiot' \
  -H 'fiware-servicepath: /' \
  -d '{
 "services": [
   {
     "apikey":      "4jggokgpepnvsb2uv4s40d59ov",
     "cbroker":     "http://orion:1026",
     "entity_type": "Thing",
     "resource":    ""
   }
 ]
}'

... and then, the provisioning of devices:

curl -iX POST \
  'http://localhost:4041/iot/devices' \
  -H 'Content-Type: application/json' \
  -H 'fiware-service: openiot' \
  -H 'fiware-servicepath: /' \
  -d '{
 "devices": [
    {
      "device_id": "door001",
      "entity_name": "urn:ngsi-ld:Door:001",
      "entity_type": "Door",
      "protocol": "PDI-IoTA-UltraLight",
      "transport": "MQTT",
      "commands": [
        {"name": "unlock","type": "command"},
        {"name": "open","type": "command"},
        {"name": "close","type": "command"},
        {"name": "lock","type": "command"}
       ],
       "attributes": [
        {"object_id": "s", "name": "state", "type":"Text"}
       ],
       "static_attributes": [
         {"name":"refStore", "type": "Relationship","value": "urn:ngsi-ld:Store:001"}
       ]
    }
  ]
}
'

[sfl0r3nz05]

I have applied next troubleshoot:

docker run -it --rm --name mqtt-publisher --network   fiware_default efrecon/mqtt-client pub -h mosquitto -m "s|ON|0|1"   -t "/ul/4jggokgpepnvsb2uv4s40d59ov/lamp001/attrs"

As you can see, it enables MQTT messages, however the lamp doesn't turn on:

[jason-fox]

Measures and commands are sent over different MQTT topics:

• Measures are sent on the /<protocol>/<api-key>/<device-id>/attrs topic
• Commands are sent on the /<api-key>/<device-id>/cmd topic.

So you sent a northbound measure, not a command. Only the IoT Agent is subscribed to the /<protocol>/<api-key>/<device-id>/attrs MQTT topic, the dummy actuator devices are subscribed to /<api-key>/<device-id>/cmd .

In this case, the on screen display you highlight is just showing all of the MQTT topics going both directions, it doesn't mean that the devices are subscribed to /attrs.

[sfl0r3nz05]

Understood, thank you !!!

Now, I am able to publish commands going southbound using POST:

curl -iX POST \
  http://localhost:4041/v2/op/update \
  -H 'Content-Type: application/json' \
  -H 'fiware-service: openiot' \
  -H 'fiware-servicepath: /' \
  -d '{
    "actionType": "update",
    "entities": [
        {
            "type": "Motion",
            "id": "urn:ngsi-ld:Motion:001",
            "on" : {
                "type": "command",
                "value": "on"
            }
        }
    ]
}'

As you can see the commandStatus is PENDING I guess because I don`t have the Context-Provider connected now.

{"id":"urn:ngsi-ld:Motion:001","type":"Motion","TimeInstant":{"type":"DateTime","value":"2022-08-15T11:08:50.640Z","metadata":{}},"count":{"type":"Integer","value":"99","metadata":{"TimeInstant":{"type":"DateTime","value":"2022-08-15T09:11:10.396Z"}}},"luminosity":{"type":"Integer","value":"335","metadata":{"TimeInstant":{"type":"DateTime","value":"2022-08-15T09:14:06.611Z"},"unitCode":{"type":"Text","value":"CDL"}}},"off_info":{"type":"commandResult","value":" ","metadata":{}},"off_status":{"type":"commandStatus","value":"UNKNOWN","metadata":{}},"on_info":{"type":"commandResult","value":" ","metadata":{}},"on_status":{"type":"commandStatus","value":"PENDING","metadata":{"TimeInstant":{"type":"DateTime","value":"2022-08-15T11:08:50.640Z"}}},"refStore":{"type":"Relationship","value":"urn:ngsi-ld:Store:001","metadata":{"TimeInstant":{"type":"DateTime","value":"2022-08-15T11:08:50.640Z"}}},"state":{"type":"Text","value":null,"metadata":{}},"on":{"type":"command","value":"","metadata":{}},"off":{"type":"command","value":"","metadata":{}}}

I will close the issue, thank you very much!!!