issuing commands from the scorpio context broker is not working

[NORTHWIND1997]

we are facing a problem when we try to issue commands to the provisioned IoT devices from the context broker. When provisioning an IoT device, the commands themselves don't show up as properties when the device entity is queried from the context broker.

Here is the problem: 1- We are using the scorpio context broker. The following docker-compose is what we use in our development environment.

version: "2"

volumes:
    mongodb: ~
    mongo-config: ~

services:

    iot-agent:
        image: fiware/iotagent-json:1.21.1-distroless
        hostname: iot-agent
        container_name: fiware-iot-agent
        depends_on:
            - mongodb
        expose:
            - "4041"
            - "7896"
        ports:
            - "4041:4041"
            - "7896:7896"
        environment:
            - "IOTA_CB_HOST=scorpio"
            - "IOTA_TIMESTAMP=true"
            - "IOTA_CB_PORT=9090"
            - "IOTA_NORTH_PORT=4041"
            - "IOTA_REGISTRY_TYPE=mongodb"
            - "IOTA_MONGO_HOST=mongodb"
            - "IOTA_MONGO_PORT=27017"
            - "IOTA_MONGO_DB=iotagent-json"
            - "IOTA_HTTP_PORT=7896"
            - "IOTA_PROVIDER_URL=http://iot-agent:4041"
            - "IOTA_CB_NGSI_VERSION=ld"
            - "IOTA_JSON_LD_CONTEXT=https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld"

    mongodb:
        image: mongo:3.2
        hostname: mongodb
        container_name: db-mongo
        ports:
            - "27017:27017"
        command: --nojournal
        volumes:
            - mongodb:/data/db
            - mongo-config:/data/configdb

    zookeeper:
        image: zookeeper
        hostname: zookeeper
        container_name: zookeeper
        networks:
            - default
        ports:
            - "2181"
        logging:
            driver: none

    kafka:
        image: bitnami/kafka
        hostname: kafka
        container_name: kafka
        networks:
            - default
        ports:
            - "9092"
        environment:
            KAFKA_ADVERTISED_HOST_NAME: kafka
            KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
            KAFKA_ADVERTISED_PORT: 9092
            KAFKA_LOG_RETENTION_MS: 10000
            KAFKA_LOG_RETENTION_CHECK_INTERVAL_MS: 5000
            ALLOW_PLAINTEXT_LISTENER: "yes"
        volumes:
            - /var/run/docker.sock:/var/run/docker.sock
        depends_on:
            - zookeeper
        logging:
            driver: none

    postgres:
        image: postgis/postgis
        hostname: postgres
        container_name: db-postgres
        networks:
            - default
        ports:
            - "5432"
        environment:
            POSTGRES_USER: ngb
            POSTGRES_PASSWORD: ngb
            POSTGRES_DB: ngb
        logging:
            driver: none

    scorpio:
        image: scorpiobroker/scorpio:scorpio-aaio-no-eureka_2.1.20
        hostname: scorpio
        container_name: fiware-scorpio
        networks:
            - default
        ports:
            - "9090:9090"
        environment:
            - context.coreurl=http://localhost:9090/ngsi-ld/contextes/ngsi-ld-core-context
        depends_on:
            - postgres
            - kafka

2- We first start by provisioning a service:

curl --location --request POST 'http://localhost:4041/iot/services' \
--header 'fiware-service: robots' \
--header 'fiware-servicepath: /' \
--header 'Content-Type: application/json' \
--data-raw '{
    "services": [{
        "apikey": "robotsapikey",
        "cbroker": "http://scorpio:9090",
        "resource": "/iot/json",
        "entity_type": "Robot"
    }]
}'

3- then we provision the IoT device. The device is set to have 2 commands (msg and count):

curl --location --request POST 'http://localhost:4041/iot/devices' \
--header 'fiware-service: robots' \
--header 'fiware-servicepath: /' \
--header 'Content-Type: application/json' \
--data-raw '{
    "devices": [
        {
            "device_id": "robot001",
            "entity_name": "robot001",
            "entity_type": "Robot",
            "transport": "HTTP",
            "endpoint": "http://host.docker.internal:8080/iot/command",
            "commands": [
                {
                    "name": "msg",
                    "type": "command"
                },
                {
                    "name": "count",
                    "type": "command"
                }
            ]
        }
    ]
}'

4- when we check for the device in the IoT agent, we see that it is there: we query using the following request

curl --location --request GET 'http://localhost:4041/iot/devices' \
--header 'fiware-service: robots' \
--header 'fiware-servicepath: /'

here is the response that we get

{
    "count": 1,
    "devices": [
        {
            "device_id": "robot001",
            "service": "robots",
            "service_path": "/",
            "entity_name": "robot001",
            "entity_type": "Robot",
            "endpoint": "http://host.docker.internal:8080/iot/command",
            "polling": false,
            "transport": "HTTP",
            "attributes": [],
            "lazy": [],
            "commands": [
                {
                    "object_id": "msg",
                    "name": "msg",
                    "type": "command"
                },
                {
                    "object_id": "count",
                    "name": "count",
                    "type": "command"
                }
            ],
            "static_attributes": [],
            "explicitAttrs": false
        }
    ]
}

5- we can even invoke commands from the IoT agent and it works fine (the IoT device is an application that we are developing in our research group that runs locally on the host machine. It's not a part of the docker-compose environment. But it works fine and responds to the commands from the IoT agent properly):

curl --location --request PATCH 'http://localhost:4041/ngsi-ld/v1/entities/robot001/attrs/msg' \
--header 'fiware-service: robots' \
--header 'fiware-servicepath: /' \
--header 'Content-Type: application/json' \
--data-raw '{
    "type": "Property",
    "value": "This is the message"
}'

6- but we don't want to issue commands or interact with the IoT agent. If we query the IoT device as an entity in the broker we get the following response: the request

curl --location --request GET 'http://localhost:9090/ngsi-ld/v1/entities/urn:ngsi-ld:Robot:robot001' \
--header 'Accept: application/ld+json' \
--header 'ngsild-Tenant: robots' \
--header 'ngsild-path: /'

the response

{
    "id": "urn:ngsi-ld:Robot:robot001",
    "type": "Robot",
    "count_info": {
        "type": "Property",
        "value": {
            "type": "commandResult",
            "@value": " "
        }
    },
    "count_status": {
        "type": "Property",
        "value": {
            "type": "commandStatus",
            "@value": "UNKNOWN"
        }
    },
    "msg_info": {
        "type": "Property",
        "value": {
            "type": "commandResult",
            "@value": "OK"
        },
        "observedAt": "2022-07-18T10:31:38.513Z"
    },
    "msg_status": {
        "type": "Property",
        "value": {
            "type": "commandStatus",
            "@value": "OK"
        },
        "observedAt": "2022-07-18T10:31:38.513Z"
    },
    "@context": [
        "https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context-v1.3.jsonld"
    ]
}

I think we all see the problem here. The entity in the broker lacks the properties for the commands. Only the status and info objects show up as attributes for the entities, but not the commands themselves. The result of this, is that we can't invoke the commands from the context broker. So if we try this:

curl --location --request PATCH 'http://localhost:9090/ngsi-ld/v1/entities/urn:ngsi-ld:Robot:robot001/attrs/msg' \
--header 'ngsild-Tenant: robots' \
--header 'ngsild-path: /' \
--header 'Content-Type: application/json' \
--data-raw '{
    "type": "Property",
    "value": "this is the message"
}'

we get the following response:

{
    "type": "https://uri.etsi.org/ngsi-ld/errors/ResourceNotFound",
    "title": "Resource not found.",
    "detail": "Provided attribute is not present"
}

Can someone tell me what the problem is ? Are we doing something wrong here ?

side note: we switched to the Orion context broker but we had the exact same problem.

[AlvaroVega]

Where is the docker-compose ?

[NORTHWIND1997]

@AlvaroVega I just updated the issue. Sorry about that I posted it by mistake before finishing it.

[AlvaroVega]

iotagent-json is currently in version 1.24, and some issues since 1.21 related with commands has been merged. Could you please try with latest version of iotagent-json ?

[NORTHWIND1997]

@AlvaroVega there is no image of version 1.24 in docker hub. Are you sure it was pushed ?

[AlvaroVega]

@AlvaroVega there is no image of version 1.24 in docker hub. Are you sure it was pushed ? This one: https://hub.docker.com/layers/iotagent-json/telefonicaiot/iotagent-json/1.24.0/images/sha256-fec24cd8faa701d238c5ad34936a633fb210f46c281db68eaefb0fd60ec6ebf5?context=repo

[NORTHWIND1997]

I just pulled this image and tested with the newer version. the problem is not fixed yet.

[fgalan]

Could you reformulate the problem in terms of the NGSIv2 API, please?

For NGSI-LD API issues, maybe @jason-fox could provide feedback or support.

[NORTHWIND1997]

The problem doesn't exist with the NGSIv2 API. I did in fact test this before as follows:

1- the environment consists of the iot-agent, the orion context broker and a mongo db:

version: "2"

volumes:
    mongodb: ~

services:
    iot-agent:
        image: fiware/iotagent-json
        hostname: iot-agent
        container_name: fiware-iot-agent
        depends_on:
            - mongodb
        expose:
            - "4041"
            - "7896"
        ports:
            - "4041:4041"
            - "7896:7896"
        environment:
            - "IOTA_CB_HOST=orion"
            - "IOTA_CB_PORT=1026"
            - "IOTA_NORTH_PORT=4041"
            - "IOTA_REGISTRY_TYPE=mongodb"
            - "IOTA_MONGO_HOST=mongodb"
            - "IOTA_MONGO_PORT=27017"
            - "IOTA_MONGO_DB=iotagent-json"
            - "IOTA_HTTP_PORT=7896"
            - "IOTA_PROVIDER_URL=http://iot-agent:4041"

    mongodb:
        image: mongo:3.2
        hostname: mongodb
        container_name: db-mongo
        ports:
            - "27017:27017"
        command: --nojournal
        volumes:
            - mongodb:/data/db

    orion:
        image: fiware/orion
        hostname: orion
        container_name: fiware-orion
        depends_on:
            - mongodb
        expose:
            - "1026"
        ports:
            - "1026:1026"
        command: -dbhost mongodb

2- the service and the device are provisioned with the same requests as in the original description (steps 2 and 3). 3- And when we query the device in the context broker we see that the command attributes are indeed present as attributes and not just the _status and _info attributes:

query using the following request:

curl --location --request GET 'http://localhost:1026/v2/entities' \
--header 'Accept: application/json' \
--header 'fiware-service: robots' \
--header 'fiware-servicepath: /'

the response we get:

[
    {
        "id": "robot001",
        "type": "Robot",
        "TimeInstant": {
            "type": "DateTime",
            "value": "2022-08-17T10:07:37.183Z",
            "metadata": {}
        },
        "count_info": {
            "type": "commandResult",
            "value": " ",
            "metadata": {}
        },
        "count_status": {
            "type": "commandStatus",
            "value": "UNKNOWN",
            "metadata": {}
        },
        "msg_info": {
            "type": "commandResult",
            "value": " ",
            "metadata": {}
        },
        "msg_status": {
            "type": "commandStatus",
            "value": "UNKNOWN",
            "metadata": {}
        },
        "msg": {
            "type": "command",
            "value": "",
            "metadata": {}
        },
        "count": {
            "type": "command",
            "value": "",
            "metadata": {}
        }
    }
]

So the problem doesn't exist when using nsgi-v2 but rather nsgi-ld.

[fgalan]

So the problem doesn't exist when using nsgi-v2 but rather nsgi-ld.

Thus, it means it is a NGSI-LD specific problem. Definitively, it would be great if @jason-fox provide feedback on this :)

[Blobonat]

For Scorpio this is the same issue as in https://github.com/telefonicaid/iotagent-ul/issues/573 (a workaround is mentioned there)

[NORTHWIND1997]

@fgalan @jason-fox is there any update on this issue ?

[mapedraza]

After closing https://github.com/telefonicaid/iotagent-ul/issues/573

I assume this issue can be closed too, right?

[jason-fox]

Yes I think so.