telefonicaid / fiware-orion

Context Broker and CEF building block for context data management, providing NGSI interfaces.
https://fiware-orion.rtfd.io/
GNU Affero General Public License v3.0
212 stars 264 forks source link

Error "400 ParseError" #3731

Open WillFreitag opened 3 years ago

WillFreitag commented 3 years ago

In our installation of Orion Context Broker we get around 0.004 % a "400 ParseError" as a response to a POST (.../v2/entities/?options=keyValues,upsert). The payload looks like:

{"id": "abc6367def", "type": "WeatherObserved", "dateObserved": {"type": "DateTime", "value": "2020-11-20T09:23:47.00Z"}, "batteryLevel": {"type": "Number", "value": 21}, "humidity": {"type": "Number", "value": 76}, "temperature": {"type": "Number", "value": 16.0}, "public": {"type": "Boolean", "value": false}, "location": {"type": "geo:json", "value": {"type": "Point", "coordinates": [2.6126363765402107, -15.724080346855551]}}}

We implemented a resend-mechanism and resending the exact same payload again get's an expected 204-response. So, for me, this is not related to the payload at all. Orion-log (log-level INFO) is not stating more than some kind of "ParseError", too.

fgalan commented 3 years ago

So, you mean that after starting the CB in a "reset" state (i.e. DB empty before starting the test) then you do this request twice in a row:

POST v2/entities/?options=keyValues,upsert

{"id": "abc6367def", "type": "WeatherObserved", "dateObserved": {"type": "DateTime", "value": "2020-11-20T09:23:47.00Z"}, "batteryLevel": {"type": "Number", "value": 21}, "humidity": {"type": "Number", "value": 76}, "temperature": {"type": "Number", "value": 16.0}, "public": {"type": "Boolean", "value": false}, "location": {"type": "geo:json", "value": {"type": "Point", "coordinates": [2.6126363765402107, -15.724080346855551]}}}

The first time you get 400 Parse Error, the second time you get 204.

Is my understanding of the case correct?

WillFreitag commented 3 years ago

Exactly this is, what we do. We send data (test-data in a high frequency) and when an 400 is encountered, we simply retry (same session, same payload) and get an 204 - on the very next time (with a short sleep (10ms) in between).

fgalan commented 3 years ago

I understand the problem only happens in high load scenarios. I mean, if you run two updates in a row in low load situation then both ones result in 204 No Content, right?

Hard to know what is going on... maybe your client has some difficulties in high load situations so sometimes it "cuts" requests sent to ContextBroker (a half-send JSON payload could explain the "ParseError")?

Maybe capturing the traffic (tcpdump, wireshark, etc.) and having a look to he request that is causing the "Parse Error" could help.