mmtk / mmtk-core

Memory Management ToolKit
https://www.mmtk.io
Other
379 stars 69 forks source link

Allow allocating without triggering GCs before `initialize_collection` #1096

Closed qinsoon closed 6 months ago

qinsoon commented 8 months ago

We should allow allocating before initialize_collection is called. See the discussion here: https://github.com/mmtk/mmtk-core/pull/1067#discussion_r1531653341

I observed a case in Julia after https://github.com/mmtk/julia/pull/39. I moved the call of initialize_collection to a later stage when Julia is actually ready for doing GCs. Depending on the default heap size, we may see the following failure when MMTk attempts to trigger a GC before Julia calls initialize_collection.

    JULIA julia-mmtk-pr/usr/lib/julia/sys-o.a
Collecting and executing precompile statements
└ Collect (Basic: thread '<unnamed>' panicked at /home/yilin/.cargo/git/checkouts/mmtk-core-3306bdeb8eb4322b/dd84218/src/policy/space.rs:103:13:
GC is not allowed here: collection is not initialized (did you call initialize_collection()?).
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
fatal runtime error: failed to initiate panic, error 5

[2268635] signal (6.-6): Aborted
in expression starting at none:0
pthread_kill at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
raise at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
abort at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
abort_internal at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/sys/unix/mod.rs:374
rust_panic at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/panicking.rs:782
rust_panic_with_hook at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/panicking.rs:753
{closure#0} at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/panicking.rs:601
__rust_end_short_backtrace<std::panicking::begin_panic_handler::{closure_env#0}, !> at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/sys_common/backtrace.rs:170
begin_panic_handler at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/panicking.rs:597
panic_fmt at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/panicking.rs:72
_ZN4mmtk6policy5space5Space7acquire17hfb334896850a4ef5E at /home/yilin/Code/julia_workspace/julia/julia-mmtk-pr/usr/lib/libmmtk_julia.so (unknown line)
_ZN4mmtk4util5alloc15immix_allocator24ImmixAllocator$LT$VM$GT$19acquire_clean_block17h55f1d14e6da964a3E at /home/yilin/Code/julia_workspace/julia/julia-mmtk-pr/usr/lib/libmmtk_julia.so (unknown line)
_ZN4mmtk4util5alloc9allocator9Allocator17alloc_slow_inline17h9b8ca11e00052653E at /home/yilin/Code/julia_workspace/julia/julia-mmtk-pr/usr/lib/libmmtk_julia.so (unknown line)
bump_alloc_fast at /home/yilin/Code/julia_workspace/julia/src/julia.h:2465 [inlined]
mmtk_immix_alloc_fast at /home/yilin/Code/julia_workspace/julia/src/julia.h:2474 [inlined]
mmtk_immix_alloc_fast at /home/yilin/Code/julia_workspace/julia/src/julia.h:2472 [inlined]
jl_mmtk_gc_alloc_default at /home/yilin/Code/julia_workspace/mmtk-julia/julia/mmtk_julia.c:53
jl_gc_alloc_ at /home/yilin/Code/julia_workspace/julia/src/julia_internal.h:494 [inlined]
ijl_box_int32 at /home/yilin/Code/julia_workspace/julia/src/datatype.c:1330
ijl_get_nth_field at /home/yilin/Code/julia_workspace/julia/src/datatype.c:1605
emit_function at /home/yilin/Code/julia_workspace/julia/src/codegen.cpp:7885
jl_emit_code at /home/yilin/Code/julia_workspace/julia/src/codegen.cpp:8657
jl_emit_codeinst at /home/yilin/Code/julia_workspace/julia/src/codegen.cpp:8731
jl_compile_workqueue at /home/yilin/Code/julia_workspace/julia/src/codegen.cpp:8862
_jl_compile_codeinst at /home/yilin/Code/julia_workspace/julia/src/jitlayers.cpp:209
jl_generate_fptr_impl at /home/yilin/Code/julia_workspace/julia/src/jitlayers.cpp:494
jl_compile_method_internal at /home/yilin/Code/julia_workspace/julia/src/gf.c:2476 [inlined]
jl_compile_method_internal at /home/yilin/Code/julia_workspace/julia/src/gf.c:2364
_jl_invoke at /home/yilin/Code/julia_workspace/julia/src/gf.c:2882 [inlined]
ijl_apply_generic at /home/yilin/Code/julia_workspace/julia/src/gf.c:3072
jl_apply at /home/yilin/Code/julia_workspace/julia/src/julia.h:2008 [inlined]
jl_module_run_initializer at /home/yilin/Code/julia_workspace/julia/src/toplevel.c:76
_finish_julia_init at /home/yilin/Code/julia_workspace/julia/src/init.c:895
julia_init at /home/yilin/Code/julia_workspace/julia/src/init.c:835
jl_repl_entrypoint at /home/yilin/Code/julia_workspace/julia/src/jlapi.c:725
main at /home/yilin/Code/julia_workspace/julia/cli/loader_exe.c:58
unknown function (ip: 0x7ff408029d8f)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
_start at /home/yilin/Code/julia_workspace/julia/julia-mmtk-pr/usr/bin/julia (unknown line)
k-sareen commented 8 months ago

Hm. Arguably, initialize_collection should be called immediately after the threading subsystem of the VM has been set up. What is Julia doing?

qinsoon commented 8 months ago

Hm. Arguably, initialize_collection should be called immediately after the threading subsystem of the VM has been set up. What is Julia doing?

Yeah. But setting up the threading system (and any other system that the threading needs) may require allocation in MMTk. If the allocation exceeds the heap size, we will see this assertion failure.

The actual problem for Julia is the call is_collection_enabled. is_collection_enabled means that the VM enables the GC, and initialize_collection means that the VM is ready to do GC. In theory, the VM should not enable GC before it is ready for GC. But currently the Julia binding returns true for is_collection_enabled before it calls initialize_collection, due to the change in https://github.com/mmtk/julia/pull/39. I am fixing this.

This is probably a binding issue, rather than a mmtk-core issue.

qinsoon commented 6 months ago

This is a binding issue, and was fixed in the Julia binding in https://github.com/mmtk/julia/pull/44.