Unusual behavior when requesting entities with specific attributes

[arturohu]

I am running a series of trials and I have detected a possible problem. The problem comes from requesting entities with a number of specific attributes.

In this case, I have established a CPr with a set of attributes for the following entity and the URL that provides it. In Orion, I have created the entity with attributes directly on it. Therefore, when requested the entity will obtain the attributes that are located in Orion and in the indicated CPr.

curl --location --request POST 'http://localhost/orion/v2/registrations' \
--header 'fiware-service: test' \
--header 'fiware-servicepath: /test' \
--header 'Content-Type: application/json' \
--data-raw '{
    "description": "Test Registration",
    "dataProvided": {
        "entities": [
            {
                "id": "urn:ngsi:Device:Test-Device1",
                "type": "Device"
            }
        ],
        "attrs": [
            "lazyAttribute1",
            "lazyAttribute2",
            "lazyAttribute3"
        ]
    },
    "provider": {
        "http": {
            "url": "http://local-ip:5000"
        }
    }
}'

curl --location --request POST 'http://localhost/orion/v2/entities' \
--header 'fiware-service: test' \
--header 'fiware-servicepath: /test' \
--header 'Content-Type: application/json' \
--data-raw '{
    "id": "urn:ngsi:Device:Test-Device1",
    "type": "Device",
    "name": {
        "metadata": {},
        "type": "Text",
        "value": "test-name"
    },
    "temperature": {
        "metadata": {},
        "type": "Number",
        "value": 35
    }
}'

The problem occurs when a specific attribute is requested which isn't in the CPr, but is in Orion, for example, the "name" attribute. I consider that it is not necessary to request anything from the CPr, even less all the lazy attributes indicated in the registration.

curl --location --request GET 'http://localhost/orion/v2/entities?attrs=name' \
--header 'fiware-service: test' \
--header 'fiware-servicepath: /test'

The test CPr receives the request from Orion. This makes performance suffer, as well as the unnecessary overload of the network. Example of a request. The CPr is asked for all lazy attributes.

POST //op/query HTTP/1.1
User-Agent: orion/2.4.0-next libcurl/7.29.0
Host: 192.168.2.139:5000
Fiware-Service: test
Fiware-Servicepath: /test
Accept: application/json
Content-Length: 129
Content-Type: application/json; charset=utf-8
Fiware-Correlator: 1e1b5fbe-ed1a-11ea-98a7-0242ac120005

{"entities":[{"id":"urn:ngsi:Device:Test-Device1","type":"Device"}],"attrs":["lazyAttribute1","lazyAttribute2","lazyAttribute3"]}

Perhaps there is a way to avoid asking the CPr or requesting only those attributes required in the query. Thank you for your support.

I have tried it on Docker:

• mongo:3.6
• fiware/orion:2.4.0 and fiware/orion:latest

[fgalan]

Thanks for the report!

In order to know the scope of this issue, could you test the following operation

GET /v2/entities/urn:ngsi:Device:Test-Device1/attrs/name

and tell us how it goes, please?

[arturohu]

Sure!

curl --location --request GET 'http://localhost/orion/v2/entities/urn:ngsi:Device:Test-Device1/attrs/name' \
--header 'fiware-service: test' \
--header 'fiware-servicepath: /test'

The test CPr receives the next request from Orion:

POST //op/query HTTP/1.1
User-Agent: orion/2.4.0-next libcurl/7.29.0
Host: 192.168.2.139:5000
Fiware-Service: test
Fiware-Servicepath: /test
Accept: application/json
Content-Length: 129
Content-Type: application/json; charset=utf-8
Fiware-Correlator: f5ba2470-eea8-11ea-bc93-0242ac150005

{"entities":[{"id":"urn:ngsi:Device:Test-Device1","type":"Device"}],"attrs":["lazyAttribute1","lazyAttribute2","lazyAttribute3"]}

Thank you!

[fgalan]

I understand the right behavior should be Orion to filter results provided by CPr to only the attributes requested by the client, in both cases (GET /v2/entities?attrs=name and GET /v2/entities/urn:ngsi:Device:Test-Device1/attrs/name)

[arturohu]

I agree with you, in fact, Orion returns only those attributes requested. But I think that in the IoT environment, and thinking about the computational, network, and resource cost of asking for attributes to remote devices, it is a problem that comes before this. I think that filtering should be done before asking CPr because this can generate unnecessary delays if the attribute for which the client is asking is not in CPr.

I will give some examples to explain this better:

• If an entity had a large number of lazy attributes, and I only asked for one of them, would it make sense to ask for all the lazy attributes to the CPr to filter in Orion only the one I asked for?

  • I think it would be more appropriate to ask only to the CPr for the attribute I asked for.

• If I had a lot of lazy attributes, and I only asked for one attribute that is not lazy and is in Orion, it would make sense to ask the CPr for all of them to filter into Orion just the one I asked for, which is furthermore none of the ones that CPr has?

  • I think it would make more sense not to ask CPr because the attribute is in Orion and it is none of the lazy ones that CPr has.

The main objective would be to reduce resource and network costs because they are very limited in this context.

Thank you very much, I hope this benefits all of us!

[fgalan]

Yes, of course. That's another way of solving the problem. Orion may ask for specific attributes (if they are known derived from the client query) when issuing the POST /v2/op/query to the Context Provider.