search

sigp / lighthouse

Ethereum consensus client in Rust
https://lighthouse.sigmaprime.io/
Apache License 2.0
2.85k stars 711 forks source link

Support page size > 4K in jemalloc #5244

Open koraykoska opened 5 months ago

koraykoska commented 5 months ago

Description

Some ARM processors have a page size other than 4k (16k for example). lighthouse bn is unable to launch on those processors, with the error: <jemalloc>: Unsupported system page size

This is a know issue as can be seen here:

https://github.com/jemalloc/jemalloc/issues/2572

As explained in the above issue, jemalloc page size is a compile time option, but higher ones are ok to use on lower page sizes, the other way round doesn't work.

My suggestion would be to increase the page size to 16k during the jemalloc compilation phase on the ARM docker images. As it's officially supported to have it higher, this shouldn't break anything, just make 16k systems work.

Version

v4.6.0

Present Behaviour

On launch of the BN: : Unsupported system page size

Expected Behaviour

Just work normally. Page size shouldn't be an issue for normal applications.

Steps to resolve

Right now the only way is to switch your kernel to a 4k page size. That's quite intrusive.

michaelsproul commented 5 months ago

I think a better option might be a feature flag that compiles jemalloc with 16K pages, which we can then decide to turn on or off by default.

I'm hesitant to turn it on by default without assessing the performance impact

koraykoska commented 5 months ago

@michaelsproul What about adding it by default to ARM builds? It's more common than 4k on ARM

michaelsproul commented 5 months ago

@koraykoska Yeah I think that would probably be fine, but we should do some benchmarking first. Let's start with an optional feature, then we can switch it on for the ARM builds by default if it proves useful.

Btw, if you build from source on the board, I think jemalloc will already autodetect the page size, won't it? So this issue is more about cross-compiling, which includes the case of building the release binary, right?

koraykoska commented 5 months ago

@michaelsproul Yes it's about the cross-compilation and especially also the docker image.

I would be happy with a special docker tag like arm-16k or something.

michaelsproul commented 5 months ago

Sounds good. I'll see if anyone from the team is interested in working on it

jtraglia commented 4 months ago

@koraykoska @michaelsproul we ran into the same issue with Reth. This was our fix: