symbol exported via --export not found: __wasm_init_memory_flag

[jozefchutka]

Using em++

emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 3.1.61 (67fa4c16496b157a7fc3377afd69ee0445e8a6e3)
clang version 19.0.0git (https:/github.com/llvm/llvm-project 7cfffe74eeb68fbb3fb9706ac7071f8caeeb6520)
Target: wasm32-unknown-emscripten

...the compilation fails when exporting ___wasm_init_memory_flag:

em++ -s EXPORTED_FUNCTIONS=_malloc,_mfcc,_free,___wasm_init_memory_flag ...
# wasm-ld: error: symbol exported via --export not found: __wasm_init_memory_flag
# em++: error: '/mfcc-wasm/emsdk/upstream/bin/wasm-ld -o wasm/mfcc.wasm /tmp/emscripten_temp_u_hbjteh/main_0.o -L/mfcc-wasm/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten -lGL-getprocaddr -lal -lhtml5 -lstubs -lc -lmimalloc -lcompiler_rt -lc++-noexcept -lc++abi-noexcept -lsockets -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr /tmp/tmpqzbnxs0dlibemscripten_js_symbols.so --strip-debug --export=malloc --export=mfcc --export=free --export=__wasm_init_memory_flag --export=_emscripten_stack_alloc --export=__get_temp_ret --export=__set_temp_ret --export=__funcs_on_exit --export=__wasm_call_ctors --export=emscripten_stack_get_current --export=_emscripten_stack_restore --export-if-defined=__start_em_asm --export-if-defined=__stop_em_asm --export-if-defined=__start_em_lib_deps --export-if-defined=__stop_em_lib_deps --export-if-defined=__start_em_js --export-if-defined=__stop_em_js --export-if-defined=fflush --export-table -z stack-size=25165824 --max-memory=4294967296 --initial-memory=26214400 --no-entry --table-base=1 --global-base=1024' failed (returned 1)

however, adding -pthread to the args, things starts working as expected and build finishes

em++ -pthread -s EXPORTED_FUNCTIONS=_malloc,_mfcc,_free,___wasm_init_memory_flag ...

I am trying to compile just a very simple project where threads are not needed, however I want to clean memory using ___wasm_init_memory_flag which works nicely once available and used.

Please make it possible to have support for ___wasm_init_memory_flag without -pthread.

[sbc100]

The __wasm_init_memory_flag symbol is really an internal detail of the linker. I doubt its worth chaning wasm-ld to add this symbol in non-pthread builds.

I see two possible solutions for you:

1) Teach you build system about the two different build mode and only try to use __wasm_init_memory_flag when building in pthreads mode 2) Use a weak reference to __wasm_init_memory_flag in your source can and export a function such as:

__attribute__((weak)) extern int* __wasm_init_memory_flag
void reset_memory_flag() {
  if (__wasm_init_memory_flag) {
    *__wasm_init_memory_flag = 0;
  }
}

I would recommend that former since its seems more precise to just not include any referecnes to that symbol except in pthread builds.

[sbc100]

Oh wait, but the memory segments themselves are active in non-pthread builds, so they only ever install themselves on module instantiation. So the __wasm_init_memory_flag technique simply doesn't work for non pthreads builds, and I don't think its likely that we can/will make it work.

[jozefchutka]

Is there any way to reset wasmMemory for non pthread builds considering __wasm_init_memory_flag is not available?

I am only interested in making non-pthread builds and would like to reuse wasmMemory instead of re-allocating it on JS side every time before running wasm.

My original idea was:

new Uint32Array(wasmMemory.buffer).fill(0); // takes up to 900ms for large memory
new Uint32Array(wasmMemory.buffer).fill(0, 0, 1024 * 1024 * 4); // ~4ms, but does it guarantee full reset?

but neither of these seems optimal to me.

[sbc100]

The short answer is no, this is not possible today.

Also, filling the memory with zeros is not going to work since there is static data that contain non-zero data on startup.

Have you measured the cost of re-instantiation? Is is very high? How are you performing the re-instantiation? I would hope that given a compiled module that the instantiation should be fast, and if its not we should focus on making that fast instead of trying to play tricks with memory.

[sbc100]

(another reason for prefering re-instantiation is that wasm modules have a other state, such as globals, and tables, and will also need to have their own reset logic).

[jozefchutka]

This is how I initiate the module:

const wasmMemory = new WebAssembly.Memory(...);
const module = await createMyModule({wasmMemory, ...});

new WebAssembly.Memory() creates zero filled memory.

For the next module creation I would like to reuse the existing wasmMemory object instead of recreating new WebAssembly.Memory()

reset(wasmMemory)
const module = await createMyModule({wasmMemory, ...});

Considering reset() would result in zero filled memory, just like new WebAssembly.Memory(), I do not understand how any globals or tables are important at all for the second createMyModule when it was all ok when zero-filled for the first instantiation?

The reset using ___wasm_init_memory_flag works and is very fast as long as it is available. But when it comes to non pthread build, even this is simpler build case, there doesn't seem to be any fast reset? At this point I am artificially adding -pthread flag to my build even it is not needed, just to be able to reset... Is this something worth to be implemented?

[sbc100]

Oh I see. If you are re-instantiating the module each time than I don't see why you would need to reset the memory at all, to be honest. The module instantiation should take care of setting up all the static data correctly during instantiation. It should work fine even without resetting __wasm_init_memory_flag (which is only needed for pthread-based builds).

[sbc100]

With non-pthread builds the memory segements are always unconditionally loaded into memory during instantiation.

For pthread-based builds only the first thread is supposed to load the memory segments, which is why it uses the __wasm_init_memory_flag flag.

[jozefchutka]

This makes sense now. Thanks for clarifying.