search

LWJGL / lwjgl3

LWJGL is a Java library that enables cross-platform access to popular native APIs useful in the development of graphics (OpenGL, Vulkan, bgfx), audio (OpenAL, Opus), parallel computing (OpenCL, CUDA) and XR (OpenVR, LibOVR, OpenXR) applications.
https://www.lwjgl.org
BSD 3-Clause "New" or "Revised" License
4.67k stars 631 forks source link

Jemalloc needs support for 16KB pages on Apple Silicon Linux #828

Closed theofficialgman closed 1 year ago

theofficialgman commented 1 year ago

Version

3.3.2 (nightly), 3.3.1, 3.3.0, 3.2.3, 3.2.2, 3.2.1, 3.2.0, 3.1.6, 3.1.5, 3.1.4, 3.1.3, 3.1.2, 3.1.1, 3.1.0, 3.0.0

Platform

Linux arm64

JDK

all JDK

Module

Jemalloc

Bug description

jemalloc needs to have the default page size raised to allow running on non 4K page size linux kernels

JEMALLOC_CONFIGURE_OPTS="--with-lg-page=16" or JEMALLOC_SYS_WITH_LG_PAGE=16

see other projects that have already implemented this: https://github.com/redis/redis/pull/11407 https://github.com/rust-lang/rust/issues/100180 https://github.com/rr-debugger/rr/pull/3146 https://lists.gnu.org/archive/html/bug-gnu-emacs/2021-03/msg01260.html https://github.com/archlinuxarm/PKGBUILDs/commit/19a1393

info at asahi linux: https://github.com/AsahiLinux/docs/wiki/Broken-Software

original commit: https://github.com/jemalloc/jemalloc/commit/da4cff0279b2e8f2b0482ae961f2e2f63662342d

Stacktrace or crash log output

No response

theofficialgman commented 1 year ago

@Spasi specifically the CI file needs changing to aarch64 to include this. similar to how mips64 has it defined already for its page size https://github.com/LWJGL-CI/jemalloc/blob/master/.github/workflows/lwjgl.yml

Spasi commented 1 year ago

Hey @theofficialgman,

The jemalloc Linux/arm64 build in 3.3.2-snapshot+8 now uses 16K pages. Thanks!

Btw, lg-page was set to 14 and not 16 because it's a base 2 log. 14 corresponds to 16K and 16 to 64K.

theofficialgman commented 1 year ago

@Spasi aarch64 page sizes can be 4K, 16K, or 64K. That's why I suggested using lg-page of 16 to be compatible with all three sizes.

theofficialgman commented 1 year ago

@Spasi please use 16 for 64K as other distros are indeed using (or trying to use) 64K kernels on all devices by default. Fedora and RHEL used 64k as default kernel page size for all aarch64 systems until very recently (where they backtracked to 4k but will likely switch again once firmware compatibility with 64k is better).

the distros that ship jemalloc as part of their packages have all switched to use 64K. Do you have reasons for using 16K? Maybe performance metrics from testing the 3 options?

Spasi commented 1 year ago

No, I haven't done any performance testing. My guess is that if 64K was universally beneficial, jemalloc devs would have made it the default. It feels sensible to use the smallest size that works and any user that knows better could build and ship a custom build with their application (LWJGL allows/encourages it).

I'm also not sure why page size is hardcoded at build time and not a runtime option.