mmtk / mmtk-core

Memory Management ToolKit
https://www.mmtk.io
Other
379 stars 69 forks source link

Don't use `PROT_EXEC` with `mmap` on macos #1110

Closed eileencodes closed 7 months ago

eileencodes commented 7 months ago

On macos I was running ./miniruby --mmtk -e 'puts "Hello world!"' and seeing a panic:

thread '<unnamed>' panicked at /src/util/raw_memory_freelist.rs:202:9:
Can't get more space with mmap()
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
fatal runtime error: failed to initiate panic, error 5
Abort trap: 6

On macos, you can't combine PROT_EXEC with PROT_WRITE and PROT_READ in mmap, it will return a permission denied error. It's not clear from the Can't get more space with mmap() that's what was happening, but I could see it when inspecting the ret from mmap_fixed in dzmmap_noreplace.

I found a thread on apple confirming that PROT_EXEC doesn't make sense for macos in this context. https://forums.developer.apple.com/forums/thread/740017

Note: there are still failing tests for mmap on macos. The main change with this PR is that it no longer panics when runnning a simple hello world with Ruby on macos. Previously the build wasn't finishing and would throw the following error:

process didn't exit successfully: `/mmtk-core/target/debug/deps/mmtk-ef0f14e36a03fbba` (signal: 11, SIGSEGV: invalid memory reference)

Now the tests do finish, with 25 failures.