search

telefonicaid / iotagent-node-lib

Module to enable IoT Agent developers to build custom agents for their devices that can easily connect to NGSI Context Brokers
https://iotagent-node-lib.rtfd.io/
GNU Affero General Public License v3.0
60 stars 88 forks source link

Reverse option sends the notification twice #1064

Open danielvillalbamota opened 3 years ago

danielvillalbamota commented 3 years ago

Environment: tutorials.IoT-Agent

IoTA version: 

{
    "libVersion": "2.15.0",
    "port": "4041",
    "baseRoot": "/",
    "version": "1.16.0"
}

CB version:

{
    "orion": {
        "version": "3.0.0",
        "uptime": "0 d, 1 h, 14 m, 22 s",
        "git_hash": "d6f8f4c6c766a9093527027f0a4b3f906e7f04c4",
        "compile_time": "Mon Apr 12 14:48:44 UTC 2021",
        "compiled_by": "root",
        "compiled_in": "f307ca0746f5",
        "release_date": "Mon Apr 12 14:48:44 UTC 2021",
        "machine": "x86_64",
        "doc": "https://fiware-orion.rtfd.io/en/3.0.0/",
        "libversions": {
            "boost": "1_66",
            "libcurl": "libcurl/7.61.1 OpenSSL/1.1.1g zlib/1.2.11 nghttp2/1.33.0",
            "libmicrohttpd": "0.9.70",
            "openssl": "1.1",
            "rapidjson": "1.1.0",
            "mongoc": "1.17.4",
            "bson": "1.17.4"
        }
    }
}

Device registration with endpoint webhook to test the commands (service group already set):

{
     "devices": [
    {
      "device_id": "tracker005",
      "entity_name": "car:005",
      "entity_type": "Car",
      "protocol": "IoTA-UL",
      "transport": "HTTP",
      "endpoint": "https://webhook.site/d897aa60-2758-43ba-9df7-f16895114220",
      "attributes": [
        {
            "object_id": "t",
            "name":"turn",
            "type":"Text",
            "expression": "${@t}",
            "reverse": [
                {
                "object_id":"t",
                "type": "Text",
                "expression": "${@turn}"
                }
            ]
        }
      ]
    }
  ]
}

When I send data from the CB: 

curl --location --request PUT 'http://localhost:1026/v2/entities/car:005/attrs/turn' \
--header 'Fiware-Service: fiware_zone' \
--header 'Fiware-ServicePath: /' \
--header 'X-Auth-Token: null' \
--header 'Content-Type: application/json' \
--data-raw '{
    "value": "left"
}'

The operation sends two different commands (seconds of difference) to the endpoint:

06/07/2021 6:35:17 PM 

tracker005@turn|left

06/07/2021 6:35:22 PM

tracker005@t|left

I would expect just one notification from the "reverse" expression, only the last command tracker005@t|left, as it is configured that t is the attribute that understand the "device". But it seems that the agent is also considering the attribute outside the reverse configuration.

danielvillalbamota commented 3 years ago

Trying to get deeper using the latest version and a simpler example I got this:

Environment:

MongoDB variables

MONGO_DB_PORT=27017 MONGO_DB_VERSION=4.4

IoT Agent Ultralight Variables

ULTRALIGHT_VERSION=latest IOTA_NORTH_PORT=4041 IOTA_SOUTH_PORT=7896 IOTA_AUTOCAST=false

Tutorial variables

TUTORIAL_APP_PORT=3000 TUTORIAL_DUMMY_DEVICE_PORT=3001


- IoTAgent-ul:

{ "libVersion": "2.15.0-next", "port": "4041", "baseRoot": "/", "version": "1.16.0-next" }

Although last tagged version is 1.16.2 the version shown by the agent is "1.16.0-next". I suppose I am using the very latest.

- `env` inside the "iot-agent" container:

YARN_VERSION=1.22.5 IOTA_LOG_LEVEL=DEBUG HOSTNAME=iot-agent IOTA_CB_PORT=1026 IOTA_TIMESTAMP=true IOTA_CB_NGSI_VERSION=v2 NODE_ENV=production IOTA_REGISTRY_TYPE=mongodb IOTA_MONGO_PORT=27017 IOTA_DEFAULT_RESOURCE=/iot/d PWD=/opt/iotaul HOME=/home/node IOTA_NORTH_PORT=4041 IOTA_AUTOCAST=false IOTA_CB_HOST=orion NODE_VERSION=12.21.0 TERM=xterm IOTA_PROVIDER_URL=http://iot-agent:4041 SHLVL=1 IOTA_HTTP_PORT=7896 IOTA_MONGO_DB=iotagentul PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin IOTA_MONGOHOST=mongo-db =/usr/bin/env


Test executed:
- Service group provision:

curl --location --request POST 'http://localhost:4041/iot/services' \ --header 'Fiware-Service: fiware_zone' \ --header 'Fiware-ServicePath: /' \ --header 'Content-Type: application/json' \ --data-raw '{ "services": [ { "attributes": [], "static_attributes": [], "protocol": [ "IoTA-UL" ],
"apikey": "4jggokgpepnvsb2uv4s40d59ov", "cbroker": "http://orion:1026", "entity_type": "bell", "resource": "/iot/d" } ] }'

- Device provision:
Entities and attributes has no real meaning.

{ "devices": [ { "device_id": "tracker008", "entity_name": "car:008", "entity_type": "Car", "protocol": "IoTA-UL", "transport": "HTTP", "endpoint": "https://webhook.site/d897aa60-2758-43ba-9df7-...", "attributes": [ { "object_id": "h", "name":"humidity", "type":"Number", "reverse": [ { "object_id":"h", "type": "Number", "expression": "${@humidity}" } ] } ] } ] }

I use webhook service to see the notifications. I keep the example simple without expressions in the attribute coming from the southbound.

I send the update on the humidity attribute through the CB:

curl --location --request PUT 'http://localhost:1026/v2/entities/car:008/attrs/humidity/value' \ --header 'Fiware-Service: fiware_zone' \ --header 'Fiware-ServicePath: /' \ --header 'X-Auth-Token: null' \ --header 'Content-Type: text/plain' \ --data-raw '24'


I get two notifications in the webhook.site:
At 06/15/2021 10:37:06 AM

curl -X 'POST' 'https://webhook.site/d897aa60-2758-43ba-9df7-...' -H 'connection: close' -H 'content-length: 22' -H 'host: webhook.site' -H 'fiware-servicepath: /' -H 'fiware-service: fiware_zone' -H 'content-type: ' -d $'tracker008@humidity|24'


And other at 06/15/2021 10:37:07 AM

curl -X 'POST' 'https://webhook.site/d897aa60-2758-43ba-9df7-...' -H 'connection: close' -H 'content-length: 15' -H 'host: webhook.site' -H 'fiware-servicepath: /' -H 'fiware-service: fiware_zone' -H 'content-type: ' -d $'tracker008@h|24'

I would expect just the last one as the notification has the attribute in the device's format.

The IoTAgent log:

... time=2021-06-15T08:37:06.411Z | lvl=DEBUG | corr=ddcc069e-cdb4-11eb-952d-0242ac120103; cbnotif=1 | trans=6cb0ac10-4935-4b52-9fe3-abaf732e4ff6 | op=IoTAgentNGSI.GenericMiddlewares | from=n/a | srv=fiware_zone | subsrv=/ | msg=Body:

{ "subscriptionId": "60c863661a8da57c325ff0fd", "data": [ { "id": "car:008", "type": "Car", "humidity": { "type": "Number", "value": 24, "metadata": {} } } ] }

| comp=IoTAgent time=2021-06-15T08:37:06.416Z | lvl=DEBUG | corr=ddcc069e-cdb4-11eb-952d-0242ac120103; cbnotif=1 | trans=6cb0ac10-4935-4b52-9fe3-abaf732e4ff6 | op=IoTAgentNGSI.ContextServer-v2 | from=n/a | srv=fiware_zone | subsrv=/ | msg=Handling notification from [iot-agent:4041] | comp=IoTAgent time=2021-06-15T08:37:06.420Z | lvl=DEBUG | corr=ddcc069e-cdb4-11eb-952d-0242ac120103; cbnotif=1 | trans=6cb0ac10-4935-4b52-9fe3-abaf732e4ff6 | op=IoTAgentNGSI.MongoDBDeviceRegister | from=n/a | srv=fiware_zone | subsrv=/ | msg=Looking for device with name [car:008]. | comp=IoTAgent time=2021-06-15T08:37:06.468Z | lvl=DEBUG | corr=2d148120-c462-42bd-a175-74ebdd5579c1 | trans=2d148120-c462-42bd-a175-74ebdd5579c1 | op=IoTAgentNGSI.MongoDBGroupRegister | from=n/a | srv=n/a | subsrv=n/a | msg=Looking for group params ["service","subservice","type","apikey"] with queryObj {"service":"fiware_zone","subservice":"/","type":"Car"} | comp=IoTAgent time=2021-06-15T08:37:06.476Z | lvl=DEBUG | corr=2d148120-c462-42bd-a175-74ebdd5579c1 | trans=2d148120-c462-42bd-a175-74ebdd5579c1 | op=IoTAgentNGSI.MongoDBGroupRegister | from=n/a | srv=n/a | subsrv=n/a | msg=Device group for fields [["service","subservice","type","apikey"]] not found: [{"service":"fiware_zone","subservice":"/","type":"Car"}] | comp=IoTAgent time=2021-06-15T08:37:06.476Z | lvl=DEBUG | corr=2d148120-c462-42bd-a175-74ebdd5579c1 | trans=2d148120-c462-42bd-a175-74ebdd5579c1 | op=IoTAgentNGSI.BidirectionalPlugin | from=n/a | srv=n/a | subsrv=n/a | msg=Processing active attributes notification | comp=IoTAgent time=2021-06-15T08:37:06.482Z | lvl=DEBUG | corr=2d148120-c462-42bd-a175-74ebdd5579c1 | trans=2d148120-c462-42bd-a175-74ebdd5579c1 | op=IoTAgentNGSI.BidirectionalPlugin | from=n/a | srv=n/a | subsrv=n/a | msg=Got the following transformations: [{"object_id":"h","type":"Number","expression":"${@humidity}"}] | comp=IoTAgent time=2021-06-15T08:37:06.483Z | lvl=DEBUG | corr=2d148120-c462-42bd-a175-74ebdd5579c1 | trans=2d148120-c462-42bd-a175-74ebdd5579c1 | op=IoTAgentNGSI.BidirectionalPlugin | from=n/a | srv=n/a | subsrv=n/a | msg=Got the following transformations: [] | comp=IoTAgent time=2021-06-15T08:37:06.488Z | lvl=DEBUG | corr=2d148120-c462-42bd-a175-74ebdd5579c1 | trans=2d148120-c462-42bd-a175-74ebdd5579c1 | op=IoTAgentNGSI.MongoDBGroupRegister | from=n/a | srv=n/a | subsrv=n/a | msg=Looking for bindings for the function [notificationHandler] and protocol [HTTP] | comp=IoTAgent time=2021-06-15T08:37:06.489Z | lvl=DEBUG | corr=2d148120-c462-42bd-a175-74ebdd5579c1 | trans=2d148120-c462-42bd-a175-74ebdd5579c1 | op=IoTAgentNGSI.MongoDBGroupRegister | from=n/a | srv=n/a | subsrv=n/a | msg=Binding found for function [notificationHandler] and protocol [HTTP] | comp=IoTAgent time=2021-06-15T08:37:06.679Z | lvl=DEBUG | corr=c04203a2-7258-4307-a3ce-f3153d36e04b | trans=c04203a2-7258-4307-a3ce-f3153d36e04b | op=IoTAgentNGSI.GenericMiddlewares | from=n/a | srv=n/a | subsrv=n/a | msg=Request for path [/iot/about] from [localhost:4041] | comp=IoTAgent time=2021-06-15T08:37:06.683Z | lvl=DEBUG | corr=c04203a2-7258-4307-a3ce-f3153d36e04b | trans=c04203a2-7258-4307-a3ce-f3153d36e04b | op=IoTAgentNGSI.DomainControl | from=n/a | srv=n/a | subsrv=n/a | msg=response-time: 5 | comp=IoTAgent time=2021-06-15T08:37:07.343Z | lvl=DEBUG | corr=ddcc069e-cdb4-11eb-952d-0242ac120103; cbnotif=1 | trans=6cb0ac10-4935-4b52-9fe3-abaf732e4ff6 | op=IoTAgentNGSI.DomainControl | from=n/a | srv=fiware_zone | subsrv=/ | msg=response-time: 944 | comp=IoTAgent ...


I see two `op=IoTAgentNGSI.BidirectionalPlugin`, but I am not sure if this is related.

If you need more information let me know.
Sunny-NEC commented 3 years ago

I am interested to work in this issue.

fgalan commented 3 years ago

@Sunny-NEC thanks for your willingness to work on this issue! Please go ahead and create the pull request with your contribution, so we can evaluate and provide feedback.

Sunny-NEC commented 3 years ago

@danielvillalbamota @fgalan I am investigation in this issue and trying to reproduce the issue.

curl -X POST -H "Fiware-Service: myHome" -H "Fiware-ServicePath: /environment" -H "Content-Type: application/json" -H "Cache-Control: no-cache" -d '{

 "devices": [
{
  "device_id": "tracker005",
  "entity_name": "car:005",
  "entity_type": "Car",
  "protocol": "IoTA-UL",
  "transport": "HTTP",
  "attributes": [
    {
        "object_id": "t",
        "name":"turn",
        "type":"Text",
        "expression": "${@t}",
        "reverse": [
            {
            "object_id":"t",
            "type": "Text",
            "expression": "${@turn}"
            }
        ]
    }
  ]
}

] }

' 'http://localhost:4041/iot/devices'

curl --location --request PUT 'http://localhost:1026/v2/entities/car:005/attrs/turn' \ --header 'Fiware-Service: fiware_zone' \ --header 'Fiware-ServicePath: /' \ --header 'X-Auth-Token: null' \ --header 'Content-Type: application/json' \ --data-raw '{ "value": "left" }'

OUTPUT : getting an error : {"error":"MethodNotAllowed","description":"method not allowed"}.

Is their any additional steps required to reproduce the issue. please let me know.

danielvillalbamota commented 3 years ago

Hello, @Sunny-NEC

I see FIWARE Service and ServicePath do not match with the provided in the Device registration.

Try using Fiware-Service: myHome and Fiware-ServicePath: /environment as the entity is in that scope.

You will also need an endpoint in the device's provision to receive the commands

Sunny-NEC commented 3 years ago

@danielvillalbamota thankyou for your response. i tried Fiware-Service: myHome and Fiware-ServicePath: /environment as the entity is in that scope. but still getting a same error {"error":"MethodNotAllowed","description":"method not allowed"}.

Blobonat commented 2 years ago

I still experience this issue in version 1.25.0

Commands are send twice during notification

mapedraza commented 2 years ago

@Blobonat can you confirm you are having this issue just when using the reverse option?

Blobonat commented 2 years ago

@mapedraza During a PoC I noticed that commands always are sent twice using following attribute configuration within the IOTA:

{
                "name": "warningOn",
                "type": "Property",
                "reverse": [
                    {
                        "object_id": "warningOn",
                        "type": "Text",
                        "expression": "${@warningOn}"
                    }
                ]
            }

Subscriptions in the NGSI-LD broker are correctly created only once.

I can't test a device setup with commands since the PoC setup is based on Scorpio, which does not support the IOTA-like actuation yet.

mapedraza commented 2 years ago

Can you provide more information about how the IoT is deployed (config, environment vars, log output, group provision etc) that helps to reproduce the issue?