FIWARE / context.Orion-LD

Context Broker and CEF building block for context data management which supports both the NGSI-LD and the NGSI-v2 APIs
https://www.etsi.org/deliver/etsi_gs/CIM/001_099/009/01.06.01_60/gs_CIM009v010601p.pdf
GNU Affero General Public License v3.0
50 stars 43 forks source link

Retrieving all Entity Types only returns the Types of the first 20 elements. #1698

Open cfreyfh opened 3 weeks ago

cfreyfh commented 3 weeks ago

When retrieving all Entity Types through the endpoint /ngsi-ld/v1/types there is a problem with the backend:

In the MongoDB query function https://github.com/FIWARE/context.Orion-LD/blob/659de323c77f03562f0b115c8f551cf88abc1b07/src/lib/orionld/mongoc/mongocEntitiesGet.cpp#L60 the URL parameters limit and offset are used for the query.

The first problem is that it is not allowed to pass the limit and offset parameters to that endpoint.

{
    "type": "https://uri.etsi.org/ngsi-ld/errors/BadRequestData",
    "title": "Unsupported URI parameter",
    "detail": "limit"
}

So only the first 20 (the default value for limit) elements are returned from the MongoDB and only the types from these entities are returned. Even if the URL parameters would be allowed, I think it makes no sense to use them in that way since the behaviour of the two parameters would be not as intended.

kzangeli commented 3 weeks ago

Ok, noted. Seems like a very silly bug! Thanks for finding it. I should have a fix real soon.

kzangeli commented 3 weeks ago

So, you are right in everything you say. Seems like I was thinking of something else when I implemented this. Makers zero sense. I'll have to rewrite the whole thing. BUT, no time for that right now, so I'll do a quick and dirty thing that at least makes you able to extract all types, with pagination, even though the limit/offset are on entities and not entity types:

  1. Allow limit/offset
  2. Set default limit to 1000 (unless set to anything else by the user)

So, in a first request you get one batch of entity types (only God knows how many). In the next "paginated" request (offset=1000) you get a second batch (might get entity types from batch 1).

Quite bad, sorry, but at least much better than it was.

We will not close this issue until I've rewritten the whole thing.

kzangeli commented 3 weeks ago

So, I think I got the library error fixed. Please test again, using the tag 1.8.0-PRE-1640 in dockerhub.

Remember it's not really fixed, just the dirty hack that I described earlier.

cfreyfh commented 3 weeks ago

Thank you, now we can get all the types as we need :-)