per-host function override of native stack requirement

[yamt]

consider that you have a few host functions which require huge native stack to run. right now, you need to bump WASM_STACK_GUARD_SIZE enough for the largest possible consumer even when you precisely know which instances/threads would use such functions. it would be nicer to provide a way to keep WASM_STACK_GUARD_SIZE small while allowing a few host functions to consume more stack so that you can use large native stack only for instances/threads which actually need it.

even within our tree, some native functions require more native stack than others. eg. https://github.com/bytecodealliance/wasm-micro-runtime/blob/d39d2ba3ca0bdc47ba81e1ee7723e6181fda3b1b/core/config.h#L451 eg. https://github.com/bytecodealliance/wasm-micro-runtime/pull/2332

possible implementation approaches:

a. extend NativeSymbol with necessary info. (eg. size_t native_stack_size)

make aot_call_function/aot_call_indirect etc detect overflows using the value.

pros:

• declarative

cons:

• NativeSymbol is not always handy when calling native function. maybe we need to copy it to eg. AOTImportFunc.
• a straightforward implementation would break the abi. (it should be ok for 2.0 though)
• direct call (what aot_compile_op_call does when signature is available) needs something different. maybe the aot compiler can emit the check. however, it would effectively make the stack requirement of the native function a part of the AOT ABI. i'm not sure if it's acceptable.

a'. wasm_runtime_set_native_api_stacks

use similar detection logic as (a), but a different api to tell stack size.

wasm_runtime_set_native_api_stacks(const char *module_name, NativeSymbolStack *native_stacks, uint32 n_symbol_stack)

cons:

• wamrc --native-lib needs extra care

b. add an extra overflow detection logic to each possible stack-hogging functions

like this:

if (!wasm_runtime_check_native_stack(exec_env, 1000000)) {
    return; /* fail */
}

pros:

• easy to implement
• flexible (eg. you can make the estimation depend on function parameters)
• consistent with aot-compiled modules. (check in callee)

cons:

• a lot of boring code changes if you have a lot of functions (eg. libc-uvwasi)
• unless we sprinkle noinline attributes, a compiler can inline functions and possibly break the check.

[wenyongh]

Hi, it seems both two methods are not so straightforward, how about we just introduce a new API to set the native stack size required for the host native APIs? Some what like:

struct NativeSymbolStack {
    const char *name;
    uint32 stack_size;
}
wasm_runtime_set_native_api_stacks(NativeSymbolStack *native_stacks, uint32 n_symbol_stack);

And the info will be propagated into a field in WASMFunctionImport and AOTFuncImport, and runtime does native stack overflow check according to the field's value before calling the native API, and by default the field value is WASM_STACK_GUARD_SIZE if it isn't set by the API. Also runtime can set the field value of some native APIs after registering some built-in native APIs.

[yamt]

Hi, it seems both two methods are not so straightforward, how about we just introduce a new API to set the native stack size required for the host native APIs? Some what like:

struct NativeSymbolStack {
    const char *name;
    uint32 stack_size;
}
wasm_runtime_set_native_api_stacks(NativeSymbolStack *native_stacks, uint32 n_symbol_stack);

And the info will be propagated into a field in WASMFunctionImport and AOTFuncImport, and runtime does native stack overflow check according to the field's value before calling the native API, and by default the field value is WASM_STACK_GUARD_SIZE if it isn't set by the API. Also runtime can set the field value of some native APIs after registering some built-in native APIs.

at this point, i don't have strong opinion on the api. (how to give the n_symbol_stack value.) your suggested api would have almost same overflow detection implementation with the first approach, right? how do you think about the last item in cons? this one:

   "direct call (what aot_compile_op_call does when signature is available)
   needs something different. maybe the aot compiler can emit the check.
   however, it would effectively make the stack requirement of the native
   function a part of the AOT ABI. i'm not sure if it's acceptable.")

[wenyongh]

Yes, it is very similar to the first approach, but no need to make changes for current runtime native libraries implementation and developer's native library registration if there is no extra requirement from him.

For the last item, yes, had better let aot compiler emit the check directly, not sure whether it is good to pass the native stack size info to wamrc like registering the empty native apis for it with wamrc --native-lib=xxx.so? We can add another option like wamrc --native-stack-lib=xxx.so?

[wenyongh]

BTW, the api had better use module name as the first argument, like wasm_runtime_register_natives: wasm_runtime_set_native_api_stacks(const char *module_name, NativeSymbolStack *native_stacks, uint32 n_symbol_stack)

[wenyongh]

Yes, it is very similar to the first approach, but no need to make changes for current runtime native libraries implementation and developer's native library registration if there is no extra requirement from him.

For the last item, yes, had better let aot compiler emit the check directly, not sure whether it is good to pass the native stack size info to wamrc like registering the empty native apis for it with wamrc --native-lib=xxx.so? We can add another option like wamrc --native-stack-lib=xxx.so?

Or just reuse the option wamrc --native-lib=xxx.so，but require the so to provide extra api to get the native stack info

[yamt]

Yes, it is very similar to the first approach, but no need to make changes for current runtime native libraries implementation and developer's native library registration if there is no extra requirement from him.

even with the first approach, users who don't care can leave it default. (similarly to what we do for NativeSymbol attachment today)

For the last item, yes, had better let aot compiler emit the check directly, not sure whether it is good to pass the native stack size info to wamrc like registering the empty native apis for it with wamrc --native-lib=xxx.so?

with the first approach, wamrc --native-lib would work as of today.

We can add another option like wamrc --native-stack-lib=xxx.so?

i'm not sure how it looks like. does xxx.so for --native-stack-lib use an api different from the current get_native_lib/init_native_lib/deinit_native_lib?

Or just reuse the option wamrc --native-lib=xxx.so，but require the so to provide extra api to get the native stack info

do you mean to export an extra symbol?

anyway, my original concern was not really about --native-lib. let me explain. by making the aot compiler emit the caller-side check for a native function, we effectively embed the corresponding n_symbol_stack value into the aot-compiled module. if we later modify the implementation of the native function and it ends up with changing the n_symbol_stack value, we need to re-aot-compile all modules which potentially use the native function. for that reason, right now, i'm inclined toward the approach (b).

[wenyongh]

Yes, it is very similar to the first approach, but no need to make changes for current runtime native libraries implementation and developer's native library registration if there is no extra requirement from him.

even with the first approach, users who don't care can leave it default. (similarly to what we do for NativeSymbol attachment today)

For the last item, yes, had better let aot compiler emit the check directly, not sure whether it is good to pass the native stack size info to wamrc like registering the empty native apis for it with wamrc --native-lib=xxx.so?

with the first approach, wamrc --native-lib would work as of today.

We can add another option like wamrc --native-stack-lib=xxx.so?

i'm not sure how it looks like. does xxx.so for --native-stack-lib use an api different from the current get_native_lib/init_native_lib/deinit_native_lib?

Or just reuse the option wamrc --native-lib=xxx.so，but require the so to provide extra api to get the native stack info

do you mean to export an extra symbol?

anyway, my original concern was not really about --native-lib. let me explain. by making the aot compiler emit the caller-side check for a native function, we effectively embed the corresponding n_symbol_stack value into the aot-compiled module. if we later modify the implementation of the native function and it ends up with changing the n_symbol_stack value, we need to re-aot-compile all modules which potentially use the native function. for that reason, right now, i'm inclined toward the approach (b).

OK, b is also good as long as it is not very trivial for you to add checks in the native apis.