```BINARYEN_FEATURES``` make no sence in settings (settings_internal.js)

[yanghaku]

I set the --enable-threads in settings_internal.js, but I still have this error:

......
  File "/opt/emsdk/upstream/emscripten/emcc.py", line 2940, in phase_emscript
    emscripten.run(in_wasm, wasm_target, final_js, memfile)
  File "/opt/emsdk/upstream/emscripten/emscripten.py", line 872, in run
    emscript(in_wasm, out_wasm, outfile_js, memfile)
  File "/opt/emsdk/upstream/emscripten/emscripten.py", line 306, in emscript
    update_settings_glue(out_wasm, metadata)
  File "/opt/emsdk/upstream/emscripten/emscripten.py", line 145, in update_settings_glue
    assert '--enable-threads' in settings.BINARYEN_FEATURES
AssertionError

And I saw the source code in emscripten.py L142, and found that the settings.BINARYEN_FEATURES has been overwritten directly. Why do this? Is this a bug?

  # start with the MVP features, and add any detected features.
  settings.BINARYEN_FEATURES = ['--mvp-features'] + metadata['features']
  if settings.ASYNCIFY == 2:
    settings.BINARYEN_FEATURES += ['--enable-reference-types']

  if settings.USE_PTHREADS:
    assert '--enable-threads' in settings.BINARYEN_FEATURES
  if settings.MEMORY64:
    assert '--enable-memory64' in settings.BINARYEN_FEATURES

[sbc100]

You shouldn't need to edit that settings_internal.js. This assert is basically saying that we expect the pthread feature to found in the metadata of the binary.

Can you share the step you used to hit this assert? Perhaps you could share at least your full link command?

[yanghaku]

I update the emsdk, and this is the new error msg. (with no settings change)

(SDK version: 3.1.22, sdk-releases-upstream-990cee04a21caafc75955d736fb45791a7f2aeee-64bit

$ ninja clean && ninja main -v
[1/1] Cleaning all built files...
Cleaning... 1 files.
[1/2] /opt/emsdk/upstream/emscripten/emcc   -std=c11 -fno-exceptions -O3 -ffunction-sections -fdata-sections -fno-rtti -fPIC   -w -Wextra -Werror  -mbulk-memory  -Wl,--strip-all,--gc-sections -sWASM=1 -sUSE_WEBGPU=1                 -sERROR_ON_UNDEFINED_SYMBOLS=0 -pthread -flto -O3 -DNDEBUG -MD -MT CMakeFiles/main.dir/main.c.o -MF CMakeFiles/main.dir/main.c.o.d -o CMakeFiles/main.dir/main.c.o -c /home/code/tests/hello_webgpu_c/main.c
[2/2] : && /opt/emsdk/upstream/emscripten/emcc -std=c11 -fno-exceptions -O3 -ffunction-sections -fdata-sections -fno-rtti -fPIC  -w -Wextra -Werror  -mbulk-memory  -Wl,--strip-all,--gc-sections -sWASM=1 -sUSE_WEBGPU=1                 -sERROR_ON_UNDEFINED_SYMBOLS=0 -pthread -flto -O3 -DNDEBUG  CMakeFiles/main.dir/main.c.o -o main.html   && :
FAILED: main.html 
: && /opt/emsdk/upstream/emscripten/emcc -std=c11 -fno-exceptions -O3 -ffunction-sections -fdata-sections -fno-rtti -fPIC                 -w -Wextra -Werror  -mbulk-memory  -Wl,--strip-all,--gc-sections -sWASM=1 -sUSE_WEBGPU=1                 -sERROR_ON_UNDEFINED_SYMBOLS=0 -pthread -flto -O3 -DNDEBUG  CMakeFiles/main.dir/main.c.o -o main.html   && :
Traceback (most recent call last):
  File "/opt/emsdk/upstream/emscripten/emcc.py", line 4151, in <module>
    sys.exit(main(sys.argv))
  File "/usr/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/opt/emsdk/upstream/emscripten/emcc.py", line 4144, in main
    ret = run(args)
  File "/opt/emsdk/upstream/emscripten/emcc.py", line 1226, in run
    phase_post_link(options, state, wasm_target, wasm_target, target)
  File "/usr/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/opt/emsdk/upstream/emscripten/emcc.py", line 2914, in phase_post_link
    phase_emscript(options, in_wasm, wasm_target, memfile)
  File "/usr/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/opt/emsdk/upstream/emscripten/emcc.py", line 2944, in phase_emscript
    emscripten.run(in_wasm, wasm_target, final_js, memfile)
  File "/opt/emsdk/upstream/emscripten/emscripten.py", line 873, in run
    emscript(in_wasm, out_wasm, outfile_js, memfile)
  File "/opt/emsdk/upstream/emscripten/emscripten.py", line 309, in emscript
    update_settings_glue(out_wasm, metadata)
  File "/opt/emsdk/upstream/emscripten/emscripten.py", line 148, in update_settings_glue
    assert '--enable-threads' in settings.BINARYEN_FEATURES
AssertionError
ninja: build stopped: subcommand failed.

[sbc100]

Is this a regression? What was the last emsdk version where it worked?

Can you try linking with EMCC_DEBUG=1 and attaching the direct output of wasm-ld (it should be named /tmp/emscripten_temp/emcc-0-base.wasm).

[yanghaku]

emsdk version: commit d0291b3216fbc9765d9cfc1a2103316b32a555c6 (HEAD -> main, tag: 3.1.22, origin/main, origin/HEAD)

compile with EMCC_DEBUG=1:

$ ninja clean && ninja main -v
[1/1] Cleaning all built files...
Cleaning... 1 files.
[1/2] /opt/emsdk/upstream/emscripten/emcc   -std=c11 -fno-exceptions -O3 -ffunction-sections -fdata-sections -fno-rtti -fPIC                 -w -Wextra -Werror  -mbulk-memory  -Wl,--strip-all,--gc-sections -sWASM=1 -sUSE_WEBGPU=1                 -sERROR_ON_UNDEFINED_SYMBOLS=0 -pthread -flto -O3 -DNDEBUG -MD -MT CMakeFiles/main.dir/main.c.o -MF CMakeFiles/main.dir/main.c.o.d -o CMakeFiles/main.dir/main.c.o -c /home/code/tests/hello_webgpu_c/main.c
config:DEBUG: emscripten config is located in /opt/emsdk/.emscripten
tools.filelock:DEBUG: Attempting to acquire lock 139682062817504 on /tmp/emscripten_temp/emscripten.lock
tools.filelock:DEBUG: Lock 139682062817504 acquired on /tmp/emscripten_temp/emscripten.lock
emcc:WARNING: invocation: /opt/emsdk/upstream/emscripten/emcc.py -std=c11 -fno-exceptions -O3 -ffunction-sections -fdata-sections -fno-rtti -fPIC -w -Wextra -Werror -mbulk-memory -Wl,--strip-all,--gc-sections -sWASM=1 -sUSE_WEBGPU=1 -sERROR_ON_UNDEFINED_SYMBOLS=0 -pthread -flto -O3 -DNDEBUG -MD -MT CMakeFiles/main.dir/main.c.o -MF CMakeFiles/main.dir/main.c.o.d -o CMakeFiles/main.dir/main.c.o -c /home/code/tests/hello_webgpu_c/main.c (in /home/code/tests/hello_webgpu_c/cmake-build-release)
profiler:DEBUG: block "parse arguments" took 0.000 seconds
shared:DEBUG: successfully executed /opt/emsdk/upstream/bin/clang --version
shared:DEBUG: sanity file up-to-date: /opt/emsdk/upstream/emscripten/cache/sanity.txt
shared:DEBUG: successfully executed /opt/emsdk/upstream/bin/llc --version
shared:INFO: (Emscripten: Running sanity checks)
profiler:DEBUG: block "sanity LLVM" took 0.000 seconds
profiler:DEBUG: block "check_sanity" took 0.116 seconds
diagnostics:DEBUG: disabled warning: linker setting ignored during compilation: 'WASM' [-Wunused-command-line-argument]
diagnostics:DEBUG: disabled warning: linker setting ignored during compilation: 'USE_WEBGPU' [-Wunused-command-line-argument]
diagnostics:DEBUG: disabled warning: linker setting ignored during compilation: 'ERROR_ON_UNDEFINED_SYMBOLS' [-Wunused-command-line-argument]
profiler:DEBUG: block "setup" took 0.000 seconds
profiler:DEBUG: block "ensure_sysroot" took 0.000 seconds
emcc:DEBUG: compiling source file: /home/code/tests/hello_webgpu_c/main.c
shared:DEBUG: successfully executed /opt/emsdk/upstream/bin/clang -target wasm32-unknown-emscripten -fignore-exceptions -fvisibility=default -D__EMSCRIPTEN_SHARED_MEMORY__=1 -DEMSCRIPTEN -D__EMSCRIPTEN_major__=3 -D__EMSCRIPTEN_minor__=1 -D__EMSCRIPTEN_tiny__=22 -Werror=implicit-function-declaration -I/opt/emsdk/upstream/emscripten/cache/sysroot/include/SDL --sysroot=/opt/emsdk/upstream/emscripten/cache/sysroot -Xclang -iwithsysroot/include/compat -std=c11 -fno-exceptions -O3 -ffunction-sections -fdata-sections -fno-rtti -fPIC -w -Wextra -Werror -mbulk-memory -pthread -flto -O3 -DNDEBUG -MD -MT CMakeFiles/main.dir/main.c.o -MF CMakeFiles/main.dir/main.c.o.d -c -matomics /home/code/tests/hello_webgpu_c/main.c -o CMakeFiles/main.dir/main.c.o
profiler:DEBUG: block "compile inputs" took 0.074 seconds
emcc:DEBUG: stopping after compile phase
diagnostics:DEBUG: disabled warning: argument unused during compilation: '--strip-all' [-Wunused-command-line-argument]
diagnostics:DEBUG: disabled warning: argument unused during compilation: '--gc-sections' [-Wunused-command-line-argument]
emcc:DEBUG: total time: 0.19 seconds
profiler:DEBUG: block "main" took 0.191 seconds
not cleaning up temp files since in debug-save mode, see them in /tmp/emscripten_temp
tools.filelock:DEBUG: Attempting to release lock 139682062817504 on /tmp/emscripten_temp/emscripten.lock
tools.filelock:DEBUG: Lock 139682062817504 released on /tmp/emscripten_temp/emscripten.lock
[2/2] : && /opt/emsdk/upstream/emscripten/emcc -std=c11 -fno-exceptions -O3 -ffunction-sections -fdata-sections -fno-rtti -fPIC                 -w -Wextra -Werror  -mbulk-memory  -Wl,--strip-all,--gc-sections -sWASM=1 -sUSE_WEBGPU=1                 -sERROR_ON_UNDEFINED_SYMBOLS=0 -pthread -flto -O3 -DNDEBUG  CMakeFiles/main.dir/main.c.o -o main.html   && :
FAILED: main.html 
: && /opt/emsdk/upstream/emscripten/emcc -std=c11 -fno-exceptions -O3 -ffunction-sections -fdata-sections -fno-rtti -fPIC                 -w -Wextra -Werror  -mbulk-memory  -Wl,--strip-all,--gc-sections -sWASM=1 -sUSE_WEBGPU=1                 -sERROR_ON_UNDEFINED_SYMBOLS=0 -pthread -flto -O3 -DNDEBUG  CMakeFiles/main.dir/main.c.o -o main.html   && :
config:DEBUG: emscripten config is located in /opt/emsdk/.emscripten
tools.filelock:DEBUG: Attempting to acquire lock 139856343411152 on /tmp/emscripten_temp/emscripten.lock
tools.filelock:DEBUG: Lock 139856343411152 acquired on /tmp/emscripten_temp/emscripten.lock
emcc:WARNING: invocation: /opt/emsdk/upstream/emscripten/emcc.py -std=c11 -fno-exceptions -O3 -ffunction-sections -fdata-sections -fno-rtti -fPIC -w -Wextra -Werror -mbulk-memory -Wl,--strip-all,--gc-sections -sWASM=1 -sUSE_WEBGPU=1 -sERROR_ON_UNDEFINED_SYMBOLS=0 -pthread -flto -O3 -DNDEBUG CMakeFiles/main.dir/main.c.o -o main.html (in /home/code/tests/hello_webgpu_c/cmake-build-release)
profiler:DEBUG: block "parse arguments" took 0.000 seconds
shared:DEBUG: successfully executed /opt/emsdk/upstream/bin/clang --version
shared:DEBUG: sanity file up-to-date: /opt/emsdk/upstream/emscripten/cache/sanity.txt
shared:DEBUG: successfully executed /opt/emsdk/upstream/bin/llc --version
shared:INFO: (Emscripten: Running sanity checks)
profiler:DEBUG: block "sanity LLVM" took 0.000 seconds
profiler:DEBUG: block "check_sanity" took 0.015 seconds
profiler:DEBUG: block "setup" took 0.000 seconds
profiler:DEBUG: block "ensure_sysroot" took 0.000 seconds
emcc:DEBUG: using object file: CMakeFiles/main.dir/main.c.o
profiler:DEBUG: block "compile inputs" took 0.000 seconds
profiler:DEBUG: block "linker_setup" took 0.000 seconds
profiler:DEBUG: block "calculate linker inputs" took 0.002 seconds
shared:DEBUG: executed /opt/emsdk/upstream/bin/llvm-nm --print-file-name /home/code/tests/hello_webgpu_c/cmake-build-release/CMakeFiles/main.dir/main.c.o
profiler:DEBUG: block "llvm_nm_multiple" took 0.014 seconds
system_libs:DEBUG: including crtbegin (crtbegin.o)
system_libs:DEBUG: including libGL (libGL-mt.a)
system_libs:DEBUG: including libal (libal.a)
system_libs:DEBUG: including libhtml5 (libhtml5.a)
system_libs:DEBUG: including libstubs (libstubs.a)
system_libs:DEBUG: including libnoexit (libnoexit.a)
system_libs:DEBUG: including libc (libc-mt.a)
system_libs:DEBUG: including libmalloc (libdlmalloc-mt.a)
system_libs:DEBUG: including libcompiler_rt (libcompiler_rt-mt.a)
system_libs:DEBUG: including libc++ (libc++-mt-noexcept.a)
system_libs:DEBUG: including libc++abi (libc++abi-mt-noexcept.a)
system_libs:DEBUG: including libsockets (libsockets-mt.a)
system_libs:DEBUG: including libwebgpu_cpp (libwebgpu_cpp-mt.a)
profiler:DEBUG: block "calculate system libraries" took 0.016 seconds
emcc:DEBUG: linking: ['--strip-all', '--gc-sections', 'CMakeFiles/main.dir/main.c.o', '-L/opt/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/lto', '/opt/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/lto/crtbegin.o', '-lGL-mt', '-lal', '-lhtml5', '-lstubs', '-lnoexit', '-lc-mt', '-ldlmalloc-mt', '-lcompiler_rt-mt', '-lc++-mt-noexcept', '-lc++abi-mt-noexcept', '-lsockets-mt', '-lwebgpu_cpp-mt']
shared:DEBUG: successfully executed /opt/emsdk/upstream/bin/wasm-ld -o main.wasm --strip-all --gc-sections CMakeFiles/main.dir/main.c.o -L/opt/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/lto /opt/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/lto/crtbegin.o -lGL-mt -lal -lhtml5 -lstubs -lnoexit -lc-mt -ldlmalloc-mt -lcompiler_rt-mt -lc++-mt-noexcept -lc++abi-mt-noexcept -lsockets-mt -lwebgpu_cpp-mt -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --import-undefined --import-memory --shared-memory --strip-debug --export-if-defined=main --export-if-defined=_emscripten_thread_init --export-if-defined=_emscripten_thread_exit --export-if-defined=_emscripten_thread_crashed --export-if-defined=_emscripten_tls_init --export-if-defined=pthread_self --export-if-defined=__start_em_asm --export-if-defined=__stop_em_asm --export-if-defined=__start_em_js --export-if-defined=__stop_em_js --export-if-defined=__main_argc_argv --export-if-defined=stackSave --export-if-defined=stackRestore --export-if-defined=stackAlloc --export-if-defined=__wasm_call_ctors --export-if-defined=__errno_location --export-if-defined=emscripten_dispatch_to_thread_ --export-if-defined=_emscripten_thread_free_data --export-if-defined=emscripten_main_browser_thread_id --export-if-defined=emscripten_main_thread_process_queued_calls --export-if-defined=emscripten_run_in_main_runtime_thread_js --export-if-defined=emscripten_stack_set_limits --export-if-defined=__get_temp_ret --export-if-defined=__set_temp_ret --export-if-defined=malloc --export-if-defined=free --export-table -z stack-size=5242880 --initial-memory=16777216 --no-entry --max-memory=16777216 --global-base=1024
profiler:DEBUG: block "link" took 0.355 seconds
emcc:DEBUG: emscript
building:DEBUG: saving debug copy /tmp/emscripten_temp/emcc-0-base.wasm
shared:DEBUG: successfully executed /opt/emsdk/upstream/bin/wasm-opt --version
shared:DEBUG: successfully executed /opt/emsdk/upstream/bin/wasm-emscripten-finalize --dyncalls-i64 --pass-arg=legalize-js-interface-exported-helpers main.wasm -o main.wasm --detect-features
building:DEBUG: saving debug copy /tmp/emscripten_temp/emcc-1-wasm-emscripten-finalize.wasm
emscripten:DEBUG: Metadata: {'asmConsts': {},
 'declares': ['wgpuInstanceRequestAdapter',
              'wgpuAdapterRequestDevice',
              'emscripten_unwind_to_js_event_loop',
              'emscripten_get_now',
              'emscripten_memcpy_big',
              '__emscripten_init_main_thread_js',
              '_emscripten_set_offscreencanvas_size',
              'emscripten_receive_on_main_thread_js',
              '_emscripten_notify_task_queue',
              'emscripten_check_blocking_allowed',
              'emscripten_date_now',
              '_emscripten_get_now_is_monotonic',
              'fd_close',
              'fd_write',
              'emscripten_resize_heap',
              'exit',
              '__emscripten_thread_cleanup',
              'fd_seek'],
 'emJsFuncs': {},
 'exports': ['__wasm_call_ctors',
             'main',
             '_emscripten_tls_init',
             'pthread_self',
             '__errno_location',
             '_emscripten_thread_init',
             '_emscripten_thread_crashed',
             'emscripten_main_thread_process_queued_calls',
             'emscripten_main_browser_thread_id',
             'emscripten_run_in_main_runtime_thread_js',
             'emscripten_dispatch_to_thread_',
             'free',
             'malloc',
             '_emscripten_proxy_execute_task_queue',
             '_emscripten_thread_free_data',
             '_emscripten_thread_exit',
             'emscripten_stack_set_limits',
             'stackSave',
             'stackRestore',
             'stackAlloc',
             'dynCall_jiji'],
 'features': [],
 'globalImports': [],
 'invokeFuncs': [],
 'mainReadsParams': False,
 'namedGlobals': {}}
profiler:DEBUG: block "get_metadata" took 0.032 seconds
building:DEBUG: saving debug copy /tmp/emscripten_temp/emcc-2-post_finalize.wasm
emscripten:DEBUG: very limited syscalls (fd_seek, fd_close, fd_write) so disabling full filesystem support
profiler:DEBUG: block "emscript" raised an exception after 0.033 seconds
profiler:DEBUG: block "post_link" raised an exception after 0.034 seconds
profiler:DEBUG: block "main" raised an exception after 0.422 seconds
Traceback (most recent call last):
  File "/opt/emsdk/upstream/emscripten/emcc.py", line 4151, in <module>
    sys.exit(main(sys.argv))
  File "/usr/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/opt/emsdk/upstream/emscripten/emcc.py", line 4144, in main
    ret = run(args)
  File "/opt/emsdk/upstream/emscripten/emcc.py", line 1226, in run
    phase_post_link(options, state, wasm_target, wasm_target, target)
  File "/usr/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/opt/emsdk/upstream/emscripten/emcc.py", line 2914, in phase_post_link
    phase_emscript(options, in_wasm, wasm_target, memfile)
  File "/usr/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/opt/emsdk/upstream/emscripten/emcc.py", line 2944, in phase_emscript
    emscripten.run(in_wasm, wasm_target, final_js, memfile)
  File "/opt/emsdk/upstream/emscripten/emscripten.py", line 873, in run
    emscript(in_wasm, out_wasm, outfile_js, memfile)
  File "/opt/emsdk/upstream/emscripten/emscripten.py", line 309, in emscript
    update_settings_glue(out_wasm, metadata)
  File "/opt/emsdk/upstream/emscripten/emscripten.py", line 148, in update_settings_glue
    assert '--enable-threads' in settings.BINARYEN_FEATURES
AssertionError
not cleaning up temp files since in debug-save mode, see them in /tmp/emscripten_temp
tools.filelock:DEBUG: Attempting to release lock 139856343411152 on /tmp/emscripten_temp/emscripten.lock
tools.filelock:DEBUG: Lock 139856343411152 released on /tmp/emscripten_temp/emscripten.lock

[sbc100]

@tlively any idea why the output binary wouln't have the threads feature enabled, even though we are linking with --shared-memory and linking against all the pthread versions of the standard libraries?

@yanghaku, Can you attach /tmp/emscripten_temp/emcc-0-base.wasm to this issue?

[yanghaku]

@sbc100 yes, this is the emscripten temp files: emscripten_temp.tar.gz

I also encountered the problem that the emscripten cannot detect the binaryen features -mbulk-memory and -msimd128 before.

[sbc100]

Hmm.. that emcc-0-base.wasm doesn't have any features section at all.

$ wasm-objdump -h emscripten_temp/emcc-0-base.wasm 

emcc-0-base.wasm:   file format wasm 0x1

Sections:

     Type start=0x0000000b end=0x0000014a (size=0x0000013f) count: 42
   Import start=0x0000014d end=0x000003b6 (size=0x00000269) count: 19
 Function start=0x000003b9 end=0x00000453 (size=0x0000009a) count: 152
    Table start=0x00000455 end=0x0000045a (size=0x00000005) count: 1
   Global start=0x0000045c end=0x0000049c (size=0x00000040) count: 12
   Export start=0x0000049f end=0x000006c2 (size=0x00000223) count: 23
    Start start=0x000006c4 end=0x000006c5 (size=0x00000001) start: 20
     Elem start=0x000006c7 end=0x000006d6 (size=0x0000000f) count: 1
DataCount start=0x000006d8 end=0x000006d9 (size=0x00000001) count: 3
     Code start=0x000006dd end=0x000079b0 (size=0x000072d3) count: 152
     Data start=0x000079b3 end=0x00007cba (size=0x00000307) count: 3

Ah.. I found the problem. Its -Wl,--strip-all .. that flag seems to be removing the sections such as features that emscripten is later depending on. Can you remove the link flag?

[yanghaku]

Yes, when I delete the --strip-all, it works! Thanks for your reply!