search

telefonicaid / fiware-orion

Context Broker and CEF building block for context data management, providing NGSI interfaces.
https://fiware-orion.rtfd.io/
GNU Affero General Public License v3.0
212 stars 265 forks source link

Entity Size PAYLOAD_MAX_SIZE #3492

Closed ptrdtznr closed 5 years ago

ptrdtznr commented 5 years ago

Hello,

we are facing some limitation in the field of robotics exchanging data. Nowadays the PAYLOAD_MAX_SIZE is 1MB (https://github.com/telefonicaid/fiware-orion/blob/8f2e12356aa023c6e55ee58bcc101c334bd5c8ef/src/lib/common/limits.h#L114), but we have to exchange data which is bigger than that.

May it be possible to increase this parameter or add it (even better) as a parameter in the docker(compose) file?

kzangeli commented 5 years ago

I looked at this a little, trying to remember why we put that limit. Long time ago but I think it was done like that just because we thought it would be enough (a little like Bill Gates :)).

I see no reason to not let users of Orion change the limit, at their own risk.

The default value should stay at 1 Mb perhaps but we should not have it hardcoded the way it is now.

I've also run so e tests on it (uppering the limit to 512 Mb) and found no problem.

It would be more than easy to add a CLI or env var to give users the freedom to change the limit.

ptrdtznr commented 5 years ago

This would be perfect! I hope we can have this change soon :)

fgalan commented 5 years ago

The limitation is explained here https://fiware-orion.readthedocs.io/en/master/user/known_limitations/index.html#request-maximum-size and it makes sense from the point of view of service denial. The larger the request size is in concurrent access scenarios, the higher the stress Orion has.

Having said that, it seems to be a good idea to make this flexible so users can change at their own risk with an extra CLI without the need of recompiling the code. The same could be done with the notification limit (see https://fiware-orion.readthedocs.io/en/master/user/known_limitations/index.html#notification-maximum-size), with an extra CLI for that (which default would be 8MB). So, go ahead! :)

In the meanwhile, if you are using docker and you can build your own image adjusted the #define PAYLOAD_MAX_SIZE on the fly before building the code in the Dockerfile could be a temporal workaround.

fgalan commented 5 years ago

Implemented by PR https://github.com/telefonicaid/fiware-orion/pull/3496