dCache / dcache

dCache - a system for storing and retrieving huge amounts of data, distributed among a large number of heterogenous server nodes, under a single virtual filesystem tree with a variety of standard access methods
https://dcache.org
277 stars 132 forks source link

nfs: calculate desired memory fraction for correct memory allocation #7529

Closed kofemann closed 3 months ago

kofemann commented 4 months ago

Motivation: Grizzly memory management uses heap memory size fraction even if direct memory is used.

https://github.com/eclipse-ee4j/grizzly/issues/2201

Moreover, for each memory slice (a memory segment per thread) allocates at least x16 chunks, which ends up at 16MB per slice (with 1MB chunk)

Modification: as dCache needs 2 * #Cores * maxIObuf memory, pre-calculate the required amount of direct memory and the corresponding fraction in relation to heap. Initialise the memory pool with 1/16 of the desired slice size to compensate memory allocator internal x16 increase.

Result: dCache starts with 256m of direct memory (we still have xroot mover)

Fixes: #7522 Acked-by: Svenja Meyer Acked-by: Lea Morschel Target: master, 9.2 Require-book: no Require-notes: yes (cherry picked from commit 8f3b984fab80beba8e42a61a3988367e4cd5ee87)