/message_store/insert cannot handle large messages

[santosj]

@hawesie we can't save the fremengrids on the mongodb using message store each message is 93MB and the maximum document size for message_store is 16MB. Any suggestion?

[ERROR] [1462353876.935001124]: Service call failed: service [/topological_exploration/save_grid] responded with an error: service cannot process request: service [/message_store/insert] responded with an error: error processing request: BSON document too large (93179595 bytes) - the connected server supports BSON document sizes up to 16777216 bytes.
[ERROR] [1462353876.935361395]: Failed to save 3D grid.

[hawesie]

Can you split it into multiple smaller messages somehow? IIRC the max message size is part of mongo itself and fixed.

[nilsbore]

I've been running into this problem lately also. Is there any way to configure mongodb to accept larger messages?

[marc-hanheide]

Hi, the message size is fixed, yes: https://docs.mongodb.com/manual/reference/limits/

Two options:

• split message up into chunks (probably not really good)
• use GridFS (in python: http://api.mongodb.com/python/current/api/gridfs/) and store the main data in GridFS and only put a reference (the file id) in the message.