Limit memory allocation when running a query

[gkorland]

Created by: @AviAvni Source: https://github.com/RedisGraph/RedisGraph/issues/2129 Arena allocator can be used to limit the amount of memory used while running queries

Brief design

each thread in the thread pool will have arena allocator instance each arena allocator will have X amount of memory declared in the config memory needed for running the query will be allocated from the arena allocator reset the arena allocator after the query run is finished

Pros

• ability to limit the memory used for query execution
• cheap memory free
• automatic memory cleanup

Cons

• the preallocated memory never freed

Open questions

• min/max memory for each allocator?
• which arena allocator to use?

[gkorland]

Origin comment by: @LiorKogan Suppose you have 32 threads. I think it is not acceptable for each thread to get 1/32 of the total RAM quota. There should probably be 'large memory' and 'small memory' threads.

I'm not sure if there is some way to estimate (or categorize) a query according to its size (do we need statistics for this?), and channel it to the most suitable thread.

Maybe simpler: to allow a single thread to allocate from 2 or more allocators when a small number of queries are actually running concurrently.

[gkorland]

Origin comment by: @LiorKogan I'll try to phrase the problem:

• We have X memory reserved for answering queries
• We have Y threads (max number of concurrent queries)
• We need to allocate memory to each query
• It is hard to know in advance how much memory each query would require
• If a query cannot get more memory, it aborts with OOM message
• We cannot delay/hold a query processing until more memory would be available

Naïve solution:

• Each query get X/Y memory

Problems with the Naïve solution:

• If, for example, only a single query is running, there is no reason not to allow it to allocate more than X/Y.

Suggested solution:

• Z arenas, each with size X/Y
• At any given time, an arena can be allocated only to one thread
• At any given time, zero or more arenas can be allocated to each thread (zero - when the thread doesn't execute any query)
• Each thread:
  • When not executing a query - gets zero arenas
  • When starts executing a query - initially gets one arena
  • When needs more memory - will get one more arena
  • Can get up to Z/2 arenas
• When all arenas are allocated:
  • new (pending) queries will wait till an arena is available
  • any existing reader that needs more memory - will fail immediately
  • any existing writer that needs more memory - will wait for an arena up to a timeout, then rollback and fail note 1: remember that different writers may write to different graphs note 2: only one writer is allowed to wait; additional writers should fail immediately (avoid 'dining philosophers')
• Once a query terminates, a thread releases all its arenas