Esbonio in Sphinx multiprocess mode eats up memory

[twodrops]

One of our users complained that on a machine with 504GB RAM and > 72 cores 😱 Esbonio takes up all the memory.

To quote our user:

We just see that Esbonio is suddenly starts spinning wild (see above), spawns dozens of sub-processes, which run malloc at full speed until no RAM is left for users. Screenshot above, when Esbonio is in "eating mode"

I found this interesting due to two reasons.

1. To see such a powerful machine where Esbonio is used :)
2. A pointer to a potential problem when esbonio.sphinx.numJobs is used in Esbonio.

For the first step, we will set esbonio.sphinx.numJobs to a fixed number and not 0 so that Sphinx/Esbonio does not start using all the available cores.

We are analysing this problem now and will get back when we have some results.

[alcarney]

on a machine with 504GB RAM and > 72 cores, Esbonio takes up all the memory.

Yikes!

We are analysing this problem now and will get back when we have some results.

I'll be interested to see what you find, simply running esbonio with esbonio.sphinx.numJobs = 0 for a bit on my laptop doesn't appear to be enough to reproduce the issue...

[twodrops]

@alcarney I realized that esbonio always starts several forked child processes. Is this normal? This is the case with esbonio.sphinx.numJobs:"1"

Tested with 0.14.3, 0.15.0

[alcarney]

Is this normal?

I'm going to say... yes

However, I don't think these are child processes, but rather background threads in the same process. htop by default will also show threads, which can be switched off by pressing H. When I do this I see only a single entry for esbonio

I am however, slightly surprised at the number of threads in your screenshot... from the top of my head I can only account for 3, the main thread (shown in white) and the 2 threads in the thread pool that pygls uses.

I don't suppose you're using any Sphinx extensions that would be spinning up threads of their own?