FIWARE / context.Orion-LD

Context Broker and CEF building block for context data management which supports both the NGSI-LD and the NGSI-v2 APIs
https://www.etsi.org/deliver/etsi_gs/CIM/001_099/009/01.06.01_60/gs_CIM009v010601p.pdf
GNU Affero General Public License v3.0
49 stars 40 forks source link

error counting entities #1557

Closed FR-ADDIX closed 3 weeks ago

FR-ADDIX commented 4 months ago

I have an error message, I would like to see the query that triggers this error, is there a way to do this?

ERROR@10:43:34 mongocEntitiesQuery.cpp[791]: Database Error (error counting entities: 17.5626500: $geoNear, $near, and $nearSphere are not allowed in this context)

I don't actually have a $near query anywhere in which I also count the entities.

ContextBroker Image: fiware/orion-ld:1.6.0-PRE-1577

kzangeli commented 4 months ago

That's weird ... Most mongo queries have traces, so, real quick, just turn on all traces and see what comes out. ( -t 0-255). If not enough, I'll add a call to LM_T for mongocEntitiesQuery. (sorry, no time, in a meeting)

FR-ADDIX commented 4 months ago

ContextBroker Image: fiware/orion-ld:1.6.0-PRE-1578 same Error

kzangeli commented 4 months ago

Did you get more info with -t 0-255?

FR-ADDIX commented 4 months ago

ok I have the trigger: a simple query with the following parameters.

http://orion-ld.fiware.svc:1026/ngsi-ld/v1/entities?type=EVChargingStation&idPattern=EVChargingStation&georel=near%3BmaxDistance==100&geometry=Point&coords=[10.178318,54.351428]
kzangeli commented 4 months ago

Perfect. The count part is an internal thing, needed for possible future pagination and distributed ops. Doesn't come from the user. I might have to rethink that part in case of geoqueries, now that as it seems it doesn't work. With this example I can take a look at how to avoid the problem. So, big thank you!!! :)

FR-ADDIX commented 4 weeks ago

Hi Ken,

we now have an application running that generates a lot of geo based queries and as a result we get a lot of error messages of type

“mongocEntitiesQuery.cpp[798]: Database Error (error counting entities: 17.5626500: $geoNear, $near, and $nearSphere are not allowed in this context)”

These messages are triggered by this type of request:

/ngsi-ld/v1/entities/?georel=near%3BmaxDistance%3D%3D301&geometry=Point&coords=[10.1348%2C54.314]&limit=300&q=name~%3DTIER'

Is there a solution to get around this problem? Our Orion version is orion-ld:1.6.0-PRE-1609

kzangeli commented 4 weeks ago

Hello,

I believe I've seen that error before. If it is what I think, it had a logical explanation. I'll search a little see if I can find it.

kzangeli commented 4 weeks ago

Found it. What I was referring to is from this very issue. Completely forgot about it, so sorry. MongoDB doesn't allow for the count request when $near is in use. It's an internal thing and I just have to avoid doing it. We'll see how the absence of the number of matching entities affects the algorithm. I've implemented the fix and functional tests are running. Now we can only pray :))

kzangeli commented 4 weeks ago

So, all tests passed - only because I have no functests depending on the entity count with georel=$near - I will get problems here some day in the future. I'll deal with that when we get there.

However, I can't merge the PR as we're having problems with the docker files, can't build. There's a lost package, can't download it. We're working on a complete modification of the dockerfiles. This will hopefully be ready later today. The PR will have to wait until then.

kzangeli commented 3 weeks ago

PR merged. Please test again and let me know, by closing the issue in case all is good, or, commenting on it if not.

FR-ADDIX commented 3 weeks ago

is this in version fiware/orion-ld:1.6.0-PRE-1615?

Then it still doesn't seem to work but there is another error message: "mongocEntitiesQuery.cpp[788]: Cannot count entities if georel is 'near'. MongoDB doesn't allow it"

kzangeli commented 3 weeks ago

Yeah, that warning, not error, is new. I need to do something to tell people it's not gonna work "completely, I can't just silently "eat" the problem.

You say "it still doesn't seem to work", what exactly do you mean with that? Is something failing, or are you just referring to the new warning?

FR-ADDIX commented 3 weeks ago

Ah ok, that's just a warning now, you're right.