search

eliasgranderubio / dagda

a tool to perform static analysis of known vulnerabilities, trojans, viruses, malware & other malicious threats in docker images/containers and to monitor the docker daemon and running docker containers for detecting anomalous activities
Apache License 2.0
1.16k stars 163 forks source link

Sort Memory errors in Mongodb #56

Closed ilcapone closed 5 years ago

ilcapone commented 5 years ago

Hi again :)

Short description

Trying to extract the reports of statistical analysis, I got this error Sort operation used more than the maximum 33554432 bytes of RAM

I could solve it by opening a shell in the mongdb container and modifying the internalQueryExecMaxBlockingSortBytes parameter to 134 MB with this command:

db.adminCommand({'setParameter':1, 'internalQueryExecMaxBlockingSortBytes':134217728})

A more in-depth description of the space restriction and how to solve it can be found here: https://mobilemonitoringsolutions.com/memory-errors-mongodb-resolve/

Persistent error

When for other reasons I had to restart the mongo database I lost the configuration and I had this error again because I did not commit to this image.

It would be possible to increase the size of mongo by default, or that somehow through the api of the dagda server configure this parameter?

Backtrace

Dagda server log error

dagda     | 2019-04-03T06:21:45.584498635Z pymongo.errors.OperationFailure: Executor error during find command :: caused by :: Sort operation used more than the maximum 33554432 bytes of RAM. Add an index, or specify a smaller limit.

Thanks in advance !!

eliasgranderubio commented 5 years ago

Hi @ilcapone ,

Before fix the issue by a mongodb admin command, could you describe me what query or analysis you were doing? Perhaps, it's necessary pagination in some API or the queries should be done in others way.

Regards.

eliasgranderubio commented 5 years ago

Closed due to inactivity.