[BUG] Community_detection does not release memory while it's processing, uses > 2x db size

[antoniofilipovic]

Memgraph version 2.10.2

Environment AWS EC2 Ubuntu 22 running memgraph-platform in a docker container

Describe the bug I run the following query on BlueJ to do community detection:

CALL community_detection.get() YIELD node, community_id WITH node, community_id WHERE 'Person' IN labels(node) SET node.community_id = community_id;

And the DB size on a fresh start uses 13 out of 28 gigs of memory

Once I run the query, it'll execute and allocate memory until all of the 28 Gb avalable is taken, and keeps going -- up until there's no memory left for allocating file / network handles, so ssh and other services become unavailable.

The only solution is to reboot the server as it'll stay in this state for multiple days

To Reproduce

Download the latest bluej db snapshop from https://bluej.memgraph.com/dump/bluej-latest.cypherl.gz and run the query on a linux host (& other platforms presumably) with 32 gb of ram. The machine will crash

Expected behavior No crash. Crash bad

Screenshots of start of query & moment the server became unavailable:

[antoniofilipovic]

This issue started from this one on Memgraph: https://github.com/memgraph/memgraph/issues/1219

[antoniofilipovic]

Issue here is that we need to add mgp_track_current_thread for all allocations happening in new threads

[gitbuda]

Most likely related memgraph/mage#219 memgraph/memgraph#955