AOT/JIT stack frame enhancement

[wenyongh]

AOT stack frame enhancement

Motivation

Currently LLVM AOT/JIT doesn’t use the stack frame like interpreter/fast-jit except in the feature of dumping call stack and performance profiling, since the operations of the function local variables and stack operators have been optimized to the operations of registers and native stack by the LLVM codegen. The stack frame data is very simple in dump-call-stack and perf-profiling features, only several fields are stored in the stack frame, including func_index, time_started, total_exec_time and total_exec_cnt. There may be some scenarios requiring to create the stack frame data like interpreter and commit data to it when needed:

• GC AOT/JIT: when doing the garbage collection, the runtime needs to traverse each stack frame in each thread and add the GC objects in the stack frame to the root set. Refer to https://github.com/bytecodealliance/wasm-micro-runtime/issues/2144.
• AOT/JIT source debugger: if the function locals and stack operands can be committed into the stack frame, it may be possible to implement the AOT source debugger like interpreter: read the local variable from stack frame, get the line number in the source file by getting the current bytecode (frame ip) from stack frame and using it to retrieve the line number in dwarf info.
• AOT/JIT Checkpoint save and restore: we may create the snapshot with all stack frame stored, and using it to restore and continue to run. Refer to https://github.com/bytecodealliance/wasm-micro-runtime/issues/2333.

The data to commit to the stack frame

It may include:

• The parameters and locals of current function
• The stack operands of current function
• The stack pointer
• The bytecode address, or instruction pointer
• The frame ref flags for GC

Some functionalities to support

• Support commit data from LLVM registers/native-stack into stack frame when branch/call opcodes are met, or when the opcodes may trigger GC object allocation
• Support commit the data from the current thread, like one thread requests to do GC
• Support commit the data from a standalone thread (non wasm thread)

And also we should support the current dump-call-stack and perf-profiling features.

Extend the AOTFrame structure

Add some fields in it, including ip, sp, frame_ref, lp and so on. And make the offset of these fields fixed, so the compiler can easily calculate the offsets of them in the running status.

[wenyongh]

https://github.com/bytecodealliance/wasm-micro-runtime/pull/2350

[tamaroning]

@wenyongh Hello, what is the current status of this issue? It seems that the dev branch has not merged to main yet. I am interested in using AOT stack frames to checkpoint/restore wasm runtimes.

[wenyongh]

Hi, the AOT stack frame feature is already supported in the main branch: it was merged into branch dev/gc_refactor, and the latter was merged into main. Please refer to: https://github.com/bytecodealliance/wasm-micro-runtime/blob/main/doc/build_wamr.md#enable-aot-stack-frame-feature

And for checkpoint/restore feature, maybe you can refer to this PR: https://github.com/bytecodealliance/wasm-micro-runtime/pull/3289

[tamaroning]

Thank you for your swift response and sharing useful information:)