search

tikv / jemallocator

Rust allocator using jemalloc as a backend
Other
332 stars 52 forks source link

Failed to increase stack memory #49

Open dertin opened 1 year ago

dertin commented 1 year ago

Why does increasing the stack memory cause an error when creating the background thread?

tikv-jemallocator = {version = "0.5", features = ["background_threads"] }

ulimit -s 536870912
./my-program

result: <jemalloc>: arena 0 background thread creation failed (11)

BusyJay commented 1 year ago

Maybe it's just too large (512MiB) for the thread to be spawn.

dertin commented 1 year ago

It only gives a problem when we have activated the features = ["background_threads"]

I tried with values of stack memory lower than 512MB and the problem is present when setting any value outside the default.

ulimit -s <bytes> this should increase the stack memory over the main function.

I tried to increase the stack memory for threads but it didn't fix the problem. RUSTFLAGS="-C link-args=-Wl,-zstack-size=<bytes>" RUST_MIN_STACK=<bytes>

What is this behavior due to?

BusyJay commented 1 year ago

When enabling background_threads, background threads are created to do purging in the background. The error code 11 is "Resource temporarily unavailable" and returned when spawning the thread in C. Not look into the code yet, but I think it's probably due to 512MB is not a good value.

dertin commented 1 year ago

There should be no problem with thread creation if there is more stack memory for the main function.

Higher stack memory values are used for applications with high recursion, among other rare cases. I think it would be good to find the cause to better understand the operation of what is happening in the back.