NGSI entity is missing attributes with manual device provisioning through Rest API

walterwootz commented 9 months ago

IoT Agent Node Lib version the issue has been seen with

3.4.4

Bound or port used (API interaction)

Northbound (Provision API and NGSI Interactions)

NGSI version

NGSIv2

Are you running a container?

No, I am running it natively

Image type

normal

Expected behaviour you didn't see

When calling POST http://localhost:4041/iot/devices a complete NGSI entity should be created, including active and lazy attributes.

Unexpected behaviour you saw

The created entity in Orion includes only lazy attributes and active ones are missing.

Steps to reproduce the problem


# 1. Provision service group

curl --location 'http://localhost:4041/iot/services' \
--header 'fiware-service: opcua_car' \
--header 'fiware-servicepath: /demo' \
--header 'Content-Type: application/json' \
--data '{
    "services": [
        {
            "apikey": "iot",
            "cbroker": "http://localhost:1026",
            "entity_type": "Device",
            "resource": "/iot/opcua"
        }
    ]
}'

# 2. Provision device

curl --location 'http://localhost:4041/iot/devices' \
--header 'fiware-service: opcua_car' \
--header 'fiware-servicepath: /demo' \
--header 'Content-Type: application/json' \
--data '{"devices": [{
  "device_id": "age09_Car",
  "entity_name": "age09_Car",
  "entity_type": "Device",
  "apikey": "iot",
  "service": "opcua_car",
  "subservice": "/demo",
  "attributes": [
    {
      "name": "EngineBrake",
      "type": "Number"
    },
    {
      "name": "Acceleration",
      "type": "Number"
    },
    {
      "name": "EngineStopped",
      "type": "Boolean"
    },
    {
      "name": "Engine_Temperature",
      "type": "Number"
    },
    {
      "name": "Engine_Oxigen",
      "type": "Number"
    }
  ],
  "lazy": [
    {
      "name": "Speed",
      "type": "Number"
    }
  ],
  "commands": [
    {
      "name": "Error",
      "type": "command"
    },
    {
      "name": "Stop",
      "type": "command"
    },
    {
      "name": "Accelerate",
      "type": "command"
    }
  ],
  "contexts": [
    {
      "id": "age09_Car",
      "type": "Device",
      "mappings": [
        {
          "ocb_id": "Events",
          "opcua_id": "ns=3;s=Events",
          "object_id": "ns=3;s=Events",
          "inputArguments": []
        },
        {
          "ocb_id": "EngineBrake",
          "opcua_id": "ns=3;s=EngineBrake",
          "object_id": "ns=3;s=EngineBrake",
          "inputArguments": []
        },
        {
          "ocb_id": "Acceleration",
          "opcua_id": "ns=3;s=Acceleration",
          "object_id": "ns=3;s=Acceleration",
          "inputArguments": []
        },
        {
          "ocb_id": "EngineStopped",
          "opcua_id": "ns=3;s=EngineStopped",
          "object_id": "ns=3;s=EngineStopped",
          "inputArguments": []
        },
        {
          "ocb_id": "Engine_Temperature",
          "opcua_id": "ns=3;s=Temperature",
          "object_id": "ns=3;s=Temperature",
          "inputArguments": []
        },
        {
          "ocb_id": "Engine_Oxigen",
          "opcua_id": "ns=3;s=Oxigen",
          "object_id": "ns=3;s=Oxigen",
          "inputArguments": []
        }
      ]
    }
  ],
  "contextSubscriptions": [
    {
      "id": "age09_Car",
      "type": "Device",
      "mappings": [
        {
          "ocb_id": "Error",
          "opcua_id": "ns=3;s=Error",
          "object_id": "ns=3;i=1000",
          "inputArguments": [
            {
              "dataType": 12,
              "type": "Error Type"
            }
          ]
        },
        {
          "ocb_id": "Speed",
          "opcua_id": "ns=3;s=Speed",
          "object_id": "ns=3;i=1000",
          "inputArguments": []
        },
        {
          "ocb_id": "Stop",
          "opcua_id": "ns=3;s=Stop",
          "object_id": "ns=3;i=1000",
          "inputArguments": []
        },
        {
          "ocb_id": "Accelerate",
          "opcua_id": "ns=3;s=Accelerate",
          "object_id": "ns=3;i=1000",
          "inputArguments": [
            {
              "dataType": 6,
              "type": "Intensity"
            }
          ]
        }
      ]
    }
  ],
  "endpoint": "opc.tcp://localhost:5001/UA/CarServer"
}
]}
'

# 3. Get entity from Orion
curl --location 'http://localhost:1026/v2/entities' \
--header 'Accept: application/json' \
--header 'Fiware-Service: opcua_car' \
--header 'Fiware-ServicePath: /demo'

Orion response:

[
    {
        "id": "age09_Car",
        "type": "Device",
        "Speed": {
            "type": "Number",
            "value": 0,
            "metadata": {}
        }
    }
]

Active attributes are missing.

Configs

{
    relaxTemplateValidation: true,
    logLevel: 'DEBUG',
    timestamp: true,
    contextBroker: {
        host: 'localhost',
        port: '1026',
        ngsiVersion: 'v2',
        jsonLdContext: 'https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld',
        service: 'opcua_car',
        subservice: '/demo'
    },
    server: {
        port: 4041
    },
    deviceRegistry: {
        type: 'mongodb'
    },
    mongodb: {
        host: 'localhost',
        port: '27017',
        db: 'iotagent_opcua'
    },
    types: {
        Device: {
            active: [
                {
                    name: 'Events',
                    type: 'Text'
                },
                {
                    name: 'EngineBrake',
                    type: 'Number'
                },
                {
                    name: 'Acceleration',
                    type: 'Number'
                },
                {
                    name: 'EngineStopped',
                    type: 'Boolean'
                },
                {
                    name: 'Engine_Temperature',
                    type: 'Number'
                },
                {
                    name: 'Engine_Oxigen',
                    type: 'Number'
                }
            ],
            lazy: [
                {
                    name: 'Speed',
                    type: 'Number'
                }
            ],
            commands: [
                {
                    name: 'Error',
                    type: 'command'
                },
                {
                    name: 'Stop',
                    type: 'command'
                },
                {
                    name: 'Accelerate',
                    type: 'command'
                }
            ]
        }
    },
    contexts: [
        {
            id: 'age01_Car',
            type: 'Device',
            mappings: [
                {
                    ocb_id: 'Events',
                    opcua_id: 'ns=3;s=Events',
                    object_id: 'ns=3;s=Events',
                    inputArguments: []
                },
                {
                    ocb_id: 'EngineBrake',
                    opcua_id: 'ns=3;s=EngineBrake',
                    object_id: 'ns=3;s=EngineBrake',
                    inputArguments: []
                },
                {
                    ocb_id: 'Acceleration',
                    opcua_id: 'ns=3;s=Acceleration',
                    object_id: 'ns=3;s=Acceleration',
                    inputArguments: []
                },
                {
                    ocb_id: 'EngineStopped',
                    opcua_id: 'ns=3;s=EngineStopped',
                    object_id: 'ns=3;s=EngineStopped',
                    inputArguments: []
                },
                {
                    ocb_id: 'Engine_Temperature',
                    opcua_id: 'ns=3;s=Temperature',
                    object_id: 'ns=3;s=Temperature',
                    inputArguments: []
                },
                {
                    ocb_id: 'Engine_Oxigen',
                    opcua_id: 'ns=3;s=Oxigen',
                    object_id: 'ns=3;s=Oxigen',
                    inputArguments: []
                }
            ]
        }
    ],
    contextSubscriptions: [
        {
            id: 'age01_Car',
            type: 'Device',
            mappings: [
                {
                    ocb_id: 'Error',
                    opcua_id: 'ns=3;s=Error',
                    object_id: 'ns=3;i=1000',
                    inputArguments: [
                        {
                            dataType: 12,
                            type: 'Error Type'
                        }
                    ]
                },
                {
                    ocb_id: 'Speed',
                    opcua_id: 'ns=3;s=Speed',
                    object_id: 'ns=3;i=1000',
                    inputArguments: []
                },
                {
                    ocb_id: 'Stop',
                    opcua_id: 'ns=3;s=Stop',
                    object_id: 'ns=3;i=1000',
                    inputArguments: []
                },
                {
                    ocb_id: 'Accelerate',
                    opcua_id: 'ns=3;s=Accelerate',
                    object_id: 'ns=3;i=1000',
                    inputArguments: [
                        {
                            dataType: 6,
                            type: 'Intensity'
                        }
                    ]
                }
            ]
        }
    ],
    events: [
        {
            ocb_id: 'Events',
            opcua_id: 'ns=3;s=Events',
            object_id: 'ns=3;s=Events',
            fields: [
                {
                    name: 'EventId',
                    type: 'ByteString'
                },
                {
                    name: 'EventType',
                    type: 'NodeId'
                },
                {
                    name: 'SourceNode',
                    type: 'NodeId'
                },
                {
                    name: 'SourceName',
                    type: 'String'
                },
                {
                    name: 'Time',
                    type: 'DateTime'
                },
                {
                    name: 'ReceiveTime',
                    type: 'DateTime'
                },
                {
                    name: 'Message',
                    type: 'LocalizedText'
                },
                {
                    name: 'Severity',
                    type: 'UInt16'
                }
            ]
        }
    ],
    service: 'opcua_car',
    subservice: '/demo',
    providerUrl: 'http://host.docker.internal:4041',
    deviceRegistrationDuration: 'P20Y',
    defaultType: 'Device',
    defaultResource: '/iot/opcua',
    explicitAttrs: false,
    extendedForbiddenCharacters: []
}

Log output

No specific error logs

AlvaroVega commented 9 months ago

Since https://github.com/telefonicaid/iotagent-node-lib/releases/tag/3.4.0 initial entity is not created, so the behavior your describe is right.

walterwootz commented 9 months ago

So what do you suggest for iot agent users? Is it better to let them create the entities manually in orion, or can we automate the creation in some way?

fgalan commented 9 months ago

According to documentation in PR https://github.com/telefonicaid/iotagent-node-lib/pull/1551 (soon to be merged)

This means that all entities into the Context Broker are created when data arrives from a device, no matter if the device is explicitly provisioned (via device provisioning API) or autoprovisioned

Does that suffice your use case? In negative case, please explain why don't.

walterwootz commented 9 months ago

Yes, the entity is created once a measure arrives from the device, but the created entity is missing active attributes. Only lazy ones are added.

fgalan commented 9 months ago

The attributes are added to the entity as they come in measures.

For instance:

First measure for a given device with data for attribute A and B is received -> the entity is created with attributes A and B
Second measure for the same device with data for attribute B and C is received -> in the enetity attribute A keeps same value, B is updated and C is added

If you need all the possible active attributes in the entity from the very beginning, please consider to pre-provision the entity at CB using the CB NGSIv2 API.

walterwootz commented 8 months ago

Ok thanks @fgalan we understand now. But there is still a problem regarding our IoTAgent OPC UA: when trying to perform provisioning through API we have tried to add a custom attribute ("mappings", see below) in the body with info about the southbound connection, but it does not get propagated to the deviceProvisioningHandler function.

POST http://localhost:4041/iot/devices

{
    "devices": [
        {
            "device_id": "age01_Car",
            "entity_name": "age01_Car",
            "entity_type": "Device",
            "apikey": "iot",
            "endpoint": "opc.tcp://localhost:5001/UA/CarServer",
            "mappings": [
                {
                    "ocb_id": "Engine_Temperature",
                    "opcua_id": "ns=3;s=Temperature",
                    "object_id": "ns=3;s=Temperature",
                    "inputArguments": []
                }
            ],
            "attributes": [
                {
                    "object_id": "Engine_Temperature",
                    "name": "Engine_Temperature",
                    "type": "Number"
                }
            ],
            "lazy": [],
            "commands": []
        }
    ]
}

AlvaroVega commented 8 months ago

Is there any way to create a tests (Proof of Concept) for iotagent-node-lib which reproduce the behavior that you report @walterwootz ? The usage of iotagent-node-lib by iotagent-json or iotagent-opcua should be equivalent, and that wrong behavior you are reporting is not observed in iotagent-json.

walterwootz commented 8 months ago

Ok thanks @fgalan we understand now. But there is still a problem regarding our IoTAgent OPC UA: when trying to perform provisioning through API we have tried to add a custom attribute ("mappings", see below) in the body with info about the southbound connection, but it does not get propagated to the deviceProvisioningHandler function.

POST http://localhost:4041/iot/devices
{
    "devices": [
        {
            "device_id": "age01_Car",
            "entity_name": "age01_Car",
            "entity_type": "Device",
            "apikey": "iot",
            "endpoint": "opc.tcp://localhost:5001/UA/CarServer",
            "mappings": [
                {
                    "ocb_id": "Engine_Temperature",
                    "opcua_id": "ns=3;s=Temperature",
                    "object_id": "ns=3;s=Temperature",
                    "inputArguments": []
                }
            ],
            "attributes": [
                {
                    "object_id": "Engine_Temperature",
                    "name": "Engine_Temperature",
                    "type": "Number"
                }
            ],
            "lazy": [],
            "commands": []
        }
    ]
}

This is the test, the "mappings" array is not forwarded to the device provisioning handler function (see the function here, line 72).

AlvaroVega commented 8 months ago

It seems that mappings is something not handled by iotagent-node-lib and maybe is only just related with opcua iotagent.

walterwootz commented 7 months ago

Resolved using internal_attributes in the provisioning body request. In that field you can pass any additional data. Thank you

telefonicaid / iotagent-node-lib