search

FIWARE / specifications

:clipboard: Open API specifications for FIWARE Generic Enablers
MIT License
29 stars 10 forks source link

Full documentation of this API #13

Open piwo1984 opened 3 years ago

piwo1984 commented 3 years ago

Our goal is to build up a context driven IoT platform for energy management. For this I'm trying to understand the NGSIv2 API for a couple of days now. But this is very difficult. Every example I can find (including the step by step guide) lacks some important points. The main problem (for now) is that I have to register serveral devices, which are able to execute commands. Until now I haven't found any description/example how to do this. What I've understand so far is:

... and there isn't any description of the registration payload. I don't know how to notify the context-broker about: "Hey CB. I'm device .... I'm able to execute these commands: ...."

Can you please update your documentation/examples.

fgalan commented 3 years ago

Can we share which documentation references (links) have you been using so far, please?

piwo1984 commented 3 years ago

Sure:

fgalan commented 3 years ago

So, as far as I understand, your goal is to build a system that acts as an IOT Agent. In that case, I'd recommend you to use the IOTA Lib. Note that library implements the IOTA-CB interface (northbound) so you will get "for free" all that stuf about registrations and /v2/op/update and /v2/op/query endpoints implementation.

There is a document on How to build and IOTA using the library in the library repository itself and probably more tutorials and documentation about it (not sure... I'm CCing @jason-fox in the case he knows more references on this).

jason-fox commented 3 years ago

A simple custom IoT Agent with working commands is described in this tutorial: https://github.com/FIWARE/tutorials.Custom-IoT-Agent/tree/master

There is an up-and-coming webinar on customizing IoT Agents scheduled for the beginning of March 2021: https://www.fiware.org/community/fiware-webinars/

piwo1984 commented 3 years ago

Thanks for this hints. Yes, in our project we have to rebuild a custom IoT Agent. For several reasons we can not use the IoT library you are providing. And for this I was searching for a concrete documentation about the communication between context-broker and iot-agent (and vice versa)

jason-fox commented 3 years ago

For NGSI-v2 you will use the /v2/registrations endpoint as described here: https://github.com/FIWARE/tutorials.Context-Providers/tree/master#seven-request

The IoT Agent is then listening on /v2/op/update an example of processing this can be found here: https://github.com/telefonicaid/iotagent-node-lib/blob/master/lib/services/northBound/contextServer-NGSI-v2.js#L55

For NGSI-LD using Orion-LD an actuation example using registration is described here: https://github.com/FIWARE/tutorials.LD-Subscriptions-Registrations/tree/master#forwarded-update

The final definition of NGSI-LD actuations within the ETSI specification is still being discussed.

piwo1984 commented 3 years ago

I still need further help. Here is my simple example: There is a shallow entity

{
    _id: {
        id: 'urn:ngsi-ld:GeoHeatStorage:001',
        type: 'GeoHeatStorage',
        servicePath: '/'
    },
    attrNames: [
        'heatStorageCapacity'
    ],
    attrs: {
        heatStorageCapacity: {
            type: 'Number',
            creDate: 1611568405,
            modDate: 1611568405,
            value: 0,
            md: {
                unitCode: {
                    type: 'Text',
                    value: 'E14'
                }
            },
            mdNames: [
                'unitCode'
            ]
        }
    },
    creDate: 1611568405,
    modDate: 1611568405,
    lastCorrelator: '2b146c10-5ef3-11eb-8013-0242ac120003'
}

For regsitering a context-provider/iot-agent I used this: POST /v2/registrations

{
  "description": "Current level of heat storage",
  "dataProvided": {
    "entities": [
      {
        "id": "urn:ngsi-ld:GeoHeatStorage:001",
        "type": "GeoHeatStorage"
      }
    ],
    "attrs": [
      "currentHeatStorageLevel"
    ]
  },
  "provider": {
      "http": {
          "url": "http://iot-agent:1880"
       },
            "supportedForwardingMode": "all"
  },
        "status": "active"
}

The resulting document within the registrations collection is:

{
    _id: ObjectId('600ec9586bcbafe46366999d'),
    description: 'Current level of heat storage',
    expiration: NumberLong(9223372036854775807),
    servicePath: '/',
    contextRegistration: [
        {
            entities: [
                {
                    id: 'urn:ngsi-ld:GeoHeatStorage:001',
                    type: 'GeoHeatStorage'
                }
            ],
            attrs: [
                {
                    name: 'currentHeatStorageLevel',
                    type: ''
                }
            ],
            providingApplication: 'http://iot-agent:1880'
        }
    ],
    status: 'active',
    format: 'normalized'
}

Know I query for this entity GET /v2/entities/urn:ngsi-ld:GeoHeatStorage:001 and get back what is within the entities collection. In the Logs I can see that the context-broker tries to access something with the correlator id (2b146c10-5ef3-11eb-8013-0242ac120003). The response is 404 . Of course there is an endpoint /op/query in the iot-agent. And now I stuck. I don't know how to debug what's going on. Is there any possibility to get the request the cb tries to execute? Can you please provide a simple example (or extend the documentation) how the cb forwards the request to the context-provider/iot-agent? But maybe there is also another issue. The registration document hasn't the same structure as yours (https://fiware-tutorials.readthedocs.io/en/latest/context-providers/index.html#context-provider-registration-actions)

fgalan commented 3 years ago

Which CB version are you using? How are you starting it (i.e. ps ax | grep contextBroker)?

piwo1984 commented 3 years ago

I'm using the docker image fiware/orion:2.4.0 and started it with /usr/bin/contextBroker -fg -multiservice -ngsiv1Autocast -dbhost context-broker-db

fgalan commented 3 years ago

Could you upgrade to fiware/orion:2.5.2 and start Orion adding -logLevel INFO to the CLI? Orion 2.5.0 introduces a re-work of the log subsystem that could be very useful to debug your forwarding scenario.

Once you do that, please repeat your test and provide the trace associated to the GET /v2/entities/urn:ngsi-ld:GeoHeatStorage:001 operation, please.

piwo1984 commented 3 years ago

The log entry for the forwarded POST is: 

time=2021-01-26T11:53:58.346Z | lvl=INFO | corr=2c6fb206-5fcd-11eb-8763-0242ac120004; cbfwd=1 | trans=1611660827-537-00000000080 | from=172.18.0.1 | srv=<none> | subsrv=<none> | comp=Orion | op=logTracing.cpp[212]:logInfoFwdRequest | msg=Request forwarded (regId: 600ec9586bcbafe46366999d): POST http://iot-agent:1880/op/query, request payload (114 bytes): {"entities":[{"id":"urn:ngsi-ld:GeoHeatStorage:001","type":"GeoHeatStorage"}],"attrs":["currentHeatStorageLevel"]}, response payload (149 bytes): <!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>Cannot POST //op/query</pre>
</body>
</html>
, response code: 404

This response is from the IoT Agent (based on Node-Red). In the log you can see that the response ist from another url (//op/query). Do you have any explanation where this additional slash comes from? There isn't any in the registration document. I also tried (with success) a curl to the target url from the bash of the running docker container.

piwo1984 commented 3 years ago

Workaround: Extend the url of the context-provider http://iot-agent:1880 => http://iot-agent:1880/v2

So I think the context-broker isn't able to query a context-provider on it's root url. You have to add a path.

fgalan commented 3 years ago

The URL used by Orion to forward requests are:

Is this different from what you have observed in your tests?

piwo1984 commented 3 years ago

As you can see in the example above I POSTed a registration with the root url (http://iot-agent:8080) for my iot agent. The agent is listening on http://iot-agent:1880/op/query and http://iot-agent:1880/op/query for POST request. I tested this. But the context-broker do not just simply add "/op/query" and "/op/update" to the provided url. It adds also a leading "/" between url and "/op/query" or "/op/update". It tries to make requests to http://iot-agent:1880//op/query. If I extend the url of the registration to http://iot-agent:1880/v2 the context-broker generates the correct url (http://iot-agent:1880/v2/op/query & http://iot-agent:1880/v2/op/update)

fgalan commented 3 years ago

What about if you use http://iot-agent:8080/ (with final slash) instead of http://iot-agent:8080 in your registration? Do CB adds double slash in that case?

(Just to fully consider all the possible cases of a potential bug in the CB)

piwo1984 commented 3 years ago

Same result with or without trailing slash: response is 404 (Cannot POST //op/query)

fgalan commented 3 years ago

I have created an issue in Orion Context Broker repository with the specific issue: https://github.com/telefonicaid/fiware-orion/issues/3768

Apart from that, anything else pending? Or can this issue (https://github.com/FIWARE/specifications/issues/13) be closed?

piwo1984 commented 3 years ago

Please extend the documentation https://swagger.lab.fiware.org/?url=https://raw.githubusercontent.com/Fiware/specifications/master/OpenAPI/ngsiv2/ngsiv2-openapi.json#/Registrations/Create%20Registration to show all possible property payload.

fgalan commented 3 years ago

Please extend the documentation https://swagger.lab.fiware.org/?url=https://raw.githubusercontent.com/Fiware/specifications/master/OpenAPI/ngsiv2/ngsiv2-openapi.json#/Registrations/Create%20Registration to show all possible property payload.

That swagger document is out of my scope, but maybe @jason-fox could move this to the people in charge of that document.