Get /v2/entities?attrs=id don't work

[fisuda]

'/v2/entities?attrs=id' doesn't seem to work in the latest version. If my understanding is mistaken, please tell me.

• Orion latest (1744eeb6a3489c2c0bfbeceadaf471d0a3784a95)

$ curl http://localhost:1026/version
{
"orion" : {
  "version" : "2.5.0-next",
  "uptime" : "0 d, 0 h, 0 m, 8 s",
  "git_hash" : "1744eeb6a3489c2c0bfbeceadaf471d0a3784a95",
  "compile_time" : "Fri Feb 12 14:55:26 UTC 2021",
  "compiled_by" : "root",
  "compiled_in" : "eea31cce414e",
  "release_date" : "Fri Feb 12 14:55:26 UTC 2021",
  "doc" : "https://fiware-orion.rtfd.io/",
  "libversions": {
     "boost": "1_53",
     "libcurl": "libcurl/7.29.0 NSS/3.53.1 zlib/1.2.7 libidn/1.28 libssh2/1.8.0",
     "libmicrohttpd": "0.9.70",
     "openssl": "1.0.2k",
     "rapidjson": "1.1.0",
     "mongodriver": "legacy-1.1.2"
  }
}
}

$ curl http://localhost:1026/v2/entities -H "Content-Type: application/json" --data '{"id":"id001"}'

$ curl http://localhost:1026/v2/entities
[{"id":"id001","type":"Thing"}]

$ curl http://localhost:1026/v2/entities?attrs=id
[]

$ curl --dump-header - "http://localhost:1026/v2/entities?attrs=id&options=count"
HTTP/1.1 200 OK
Connection: Keep-Alive
Content-Length: 2
Content-Type: application/json
Fiware-Total-Count: 1
Fiware-Correlator: f4e21862-70fa-11eb-bf5e-0242c0a8800e
Date: Wed, 17 Feb 2021 08:34:31 GMT

[]

• Orion 2.5.2 (11e4cbfef30d28347162e5c4ef4de3a5d2797c69)

$ curl http://localhost:1026/version
{
"orion" : {
  "version" : "2.5.2",
  "uptime" : "0 d, 0 h, 1 m, 18 s",
  "git_hash" : "11e4cbfef30d28347162e5c4ef4de3a5d2797c69",
  "compile_time" : "Thu Dec 17 08:43:46 UTC 2020",
  "compiled_by" : "root",
  "compiled_in" : "5a4a8800b1fa",
  "release_date" : "Thu Dec 17 08:43:46 UTC 2020",
  "doc" : "https://fiware-orion.rtfd.io/en/2.5.2/",
  "libversions": {
     "boost": "1_53",
     "libcurl": "libcurl/7.29.0 NSS/3.53.1 zlib/1.2.7 libidn/1.28 libssh2/1.8.0",
     "libmicrohttpd": "0.9.70",
     "openssl": "1.0.2k",
     "rapidjson": "1.1.0",
     "mongodriver": "legacy-1.1.2"
  }
}
}

$ curl http://localhost:1026/v2/entities -H "Content-Type: application/json" --data '{"id":"id001"}'

$ curl http://localhost:1026/v2/entities
[{"id":"id001","type":"Thing"}]

$ curl http://localhost:1026/v2/entities?attrs=id
[{"id":"id001","type":"Thing"}]

Thanks

[fgalan]

I have one doubt, please:

Which is the intended use case for using GET /v2/entities?attrs=id? To get a list of just entity id and type for all entities, filtering out any attribute?

Thanks for the feedback!

[fisuda]

Thank you for your reply. I'm developing a command-line interface for NGSIv2.

The following command prints a list of IDs of all entities. It's like ls command. It uses GET /v2/entities?attrs=id.

$ ngsi list --host orion entities
airqualityobserved_0
airqualityobserved_1
airqualityobserved_2
airqualityobserved_3
airqualityobserved_4
airqualityobserved_5
airqualityobserved_6

This command prints id and attributes when running with --verbose,

$ ngsi list --host orion entities --verbose 
[{"id":"airqualityobserved_0","type":"AirQualityObserved","CO":{"type":"Number","value":325.891791155,"metadata":{}},"CO_Level":{"type":"Text","value":"low","metadata":{}},"NO":{"type":"Number","value":67.860296619,"metadata":{}},"NO2":{"type":"Number","value":51.948542112,"metadata":{}},"NOx":{"type":"Number","value":

The following command deletes all entities of certain entity type.

$ ngsi rm --host orion --type Thing

To get list of entities that will be deleted, I use GET /v2/entities?attrs=id&type={type} Even in this case, I can't get a list of IDs in the latest version.

$ curl http://localhost:1026/v2/entities -H "Content-Type: application/json" --data '{"id":"id001"}'

$ curl "http://localhost:1026/v2/entities?attrs=id&type=Thing"
[]

I want to reduce a size of the payload without getting unnecessary attributes.

Thank you.

[fgalan]

I think the change of behavior was introduced by PR https://github.com/telefonicaid/fiware-orion/pull/3751. In particular by this modification:

That modification was aimed at improving responses in some query forwarding scenarios but it seems (your feedback proves it) that also have a lateral effect impacting on how attrs= behave in scenarios without forwarding.

In fact, we should have detected it with the test/functionalTest/cases/2596_retrieve_entity_without_attrs/retrieve_entity_without_attrs.test and test/functionalTest/cases/2604_op_query_attrs/op_query_attrs.test tests, but we didn't realize (the PR includes modification in these tests and it shouldn't). Probably due to a missconcention between q=A (the parameters used to filter, i.e. to decide which entities to include in the response) and attrs=A (the parameter use to project, i.e. to decide which attributes, of the selected entities, are included in the response).

I'll have a look and try to modify the implementation, to keep the previous behavior with regards to attrs= in scenarios without forwarding and, at the same time, improve the responses to query forwarding in the intended way.

As side-note: it's interesting the usage you do of attrs=, using attrs=id. The NGSIv2 spec specifies that attrs= has to be a "comma-separated list of attribute names whose data are to be included in the response". But id is not a valid name for an attribute (from "JSON Entity Representation" section: "id and type are not allowed to be used as attribute names"). So with attrs=id you ensure that no attribute will match and you will always retrieve only the id and type of the entity. I think is a good solution by the time being, but maybe in the future we should define an special keyword to specify "I don't want any attribute" in a more explicit way, i.e. attrs=! (similar to the attrs=* we have now specify "all attributes").

[kzangeli]

attrs=! sounds like a good solution. In my opinion, if id (or any other 'non-attribute', like type or servicePath) is included in the attrs list of attribute names, the broker should give an error ...

[fisuda]

Thank you for the detailed explanation. I didn't understand that id is not a valid name for an attribute. My doubt was clear up.

[fgalan]

Thank you for the detailed explanation. I didn't understand that id is not a valid name for an attribute. My doubt was clear up.

While something like attrs=! gets implemented (I have create an issue about it: https://github.com/telefonicaid/fiware-orion/issues/3780), maybe a better alternative is to use something like attrs=__NONE which is a valid attr name (from the point of view of NGSIv2 spec rules) and will suffice for your use case on getting only entity ids and types in the response.

[fgalan]

Anyway, this fix in Orion is required:

I'll have a look and try to modify the implementation, to keep the previous behavior with regards to attrs= in scenarios without forwarding and, at the same time, improve the responses to query forwarding in the intended way.

Otherwise, the result with attrs=__NONE will be the same than with attrs=id (i.e. no entities in the respone).

[fisuda]

I updated my tool (NGSI Go). It uses GET /v2/entities?attrs=__NONE to get a list of entity IDs. Related PR: https://github.com/lets-fiware/ngsi-go/pull/88

[fisuda]

Cc: @jason-fox I sent a PR (https://github.com/FIWARE/tutorials.CRUD-Operations/pull/15) to update FIWARE/tutorials.CRUD-Operations.

[fgalan]

Fixed by PR https://github.com/telefonicaid/fiware-orion/pull/3783

[fgalan]

PR with the fix already merged in master.

@fisuda could you check with Orion latest (at commit 3984f9f or later) if the problem is now fixed and GET /v2/entities?attrs=__NONE correctly shows the id and type of all entities, please?

[fisuda]

GET /v2/entities?attrs=__NONE worked as expected. This issue has been resolved. I appreciate your prompt response to fix it.

$ ngsi --version
ngsi version 0.6.0-next (git_hash:30e8e5338f7f863420e4b06f0acee1b75e26efeb)

$ ngsi version --host orion
{
"orion" : {
  "version" : "2.5.0-next",
  "uptime" : "0 d, 0 h, 3 m, 39 s",
  "git_hash" : "3984f9fc30e90fa04682131ca4516b4d277eb27e",
  "compile_time" : "Mon Feb 22 17:37:11 UTC 2021",
  "compiled_by" : "root",
  "compiled_in" : "5e9c61ba627a",
  "release_date" : "Mon Feb 22 17:37:11 UTC 2021",
  "doc" : "https://fiware-orion.rtfd.io/",
  "libversions": {
     "boost": "1_53",
     "libcurl": "libcurl/7.29.0 NSS/3.53.1 zlib/1.2.7 libidn/1.28 libssh2/1.8.0",
     "libmicrohttpd": "0.9.70",
     "openssl": "1.0.2k",
     "rapidjson": "1.1.0",
     "mongodriver": "legacy-1.1.2"
  }
}
}

$ ngsi create entity --data '{"id":"id001","A":{"type":"Text","value":"FIWARE"}}'

$ ngsi list entities
id001

$ curl http://orion:1026/v2/entities?attrs=__NONE
[{"id":"id001","type":"Thing"}]

$ ngsi create entity --data '{"id":"id002","type":"Event","A":{"type":"Text","value":"FIWARE"}}'

$ ngsi list entities
id001
id002

$ ngsi list entities --verbose
[{"id":"id001","type":"Thing","A":{"type":"Text","value":"FIWARE","metadata":{}}},{"id":"id002","type":"Event","A":{"type":"Text","value":"FIWARE","metadata":{}}}]

$ ngsi rm --type Thing --run
1

$ ngsi list entities
id002

$ ngsi rm --type Event --run
1

$ ngsi list entities