Dynamic linking emscripten::val(..) fails in SIDE_MODULE

lpotter commented 2 years ago

emscripten 3.1.7

Error message: BindingError: emval::as has unknown type N10emscripten3valE

Using emscripten::val("some text") in a SIDE_MODULE fails under some circumstances. Other uses of emscripten::val work, just not the c'tor.

Fails: emscripten::val rawPlatform = emscripten::val::global("navigator")["platform"]; if (rawPlatform.call("includes", emscripten::val("Mac")))

Succeeds: emscripten::val rawPlatform = emscripten::val::global("navigator")["platform"]; if (rawPlatform.call("includes", std::string("Mac")))

Both work fine in non dynamic linking build, or if invoked from MAIN_MODULE.

I tried a simple example app, but it was not failing there. But fails in a big project - Qt.

Also, we use MODULARIZE and EXPORT_NAME

Any comments, hints, or suggestions welcome.

sbc100 commented 2 years ago

Would you mind posting a full example that fails? I imagine this is because the static constructor generated by EMSCRIPTEN_BINDINGS is not being run when side module is loaded (for some reason).

lpotter commented 2 years ago

The codebase in which it fails is rather large - Qt's qtbase. I have not been able to create a smaller example in which it fails. If you just want the failing binaries somewhere to run, I can upload them somewhere.

Another error message is:

Uncaught (in promise) RuntimeError: Aborted(Assertion failed: Missing signature argument to addFunction: function __emval_new(handle, argCount, argTypes, args) { handle = Emval.toValue(handle);

  var newer = emval_newers[argCount];
  if (!newer) {
      newer = craftEmvalAllocator(argCount);
      emval_newers[argCount] = newer;
  }

  return newer(handle, argTypes, args);
})

There are several SIDE_MODULES which depend on other SIDE_MODULES.

Here is the example. Warning, the download total is more than 300 mb (debug build) http://llornkcor.com/qt/dynamic/qtcore_dynamic.html

lpotter commented 2 years ago

Here are the console messages with LIBRARY_DEBUG=1 https://pastebin.com/pUJGnG76

Last line being: 11:16:58.791 [library call:__emval_as: 0x8 (8),0x91c4e8 (9553128),0x565270 (5657200)] qtloader.js:396:25

and it is failing on .as<emscripten::val>();

kripken commented 2 years ago

@lpotter

I hope we get someone with embind expertise to look at more embind bugs soon, but for now, if this is important for you then I think a minimal testcase could make this much more accessible even for non-embind experts (like @sbc100 and myself). Specifically, it would be good to have a small testcase that can be built from source, that clearly shows the issue. Extracting the relevant parts out of your entire app into a small testcase would be what I recommend. (The best thing possible would be to extend an existing testcase in our test suite to show the problem.)

Separately, it may be worth trying some general things like building with sanitizers - those can catch bugs both in your application and in emscripten system code.

lpotter commented 2 years ago

Thanks for responding. Small test cases I come up with seem to be working fine. The actual error message is: BindingError: emval::as has unknown type N10emscripten3valE

Perhaps this has to do with c++ mangling.

lpotter commented 1 year ago

So we have found a workaround for us, and I have not dug into the actual cause yet. The problem goes away when we do not use MODULARIZE. We were using MODULARIZE and setting the module name with a custom loader and shell html. So the issue may be within our custom loader or with MODULARIZE within Emscripten.

torarnv commented 1 year ago

In my tests I'm seeing the same issue with a non-MODULARIZE'ed app.

torarnv commented 1 year ago

We're hitting this:

https://github.com/emscripten-core/emscripten/pull/2286/files#diff-9e4cc1973ede37b961d9f900e3abd6befa0e62eba7d51a8229d49a3a3ff31074L23

@chadaustin Any thoughts on whether the current code can be easily fixed to dealing with this in a shared library situation?

torarnv commented 1 year ago

This works around the issue in embind, without requiring stable TypeIDs across libraries:

https://gist.github.com/torarnv/f591e143c210b46560bc5418b5d8a25b

Will add a test-case to verify this outside of our code.

g1204 commented 1 year ago

try build main module with -s EXPORTED_FUNCTIONS="['__ZTIN10emscripten3valE']" It fixed my 'unbound types: N10emscripten3valE error'

torarnv commented 1 year ago

Good tip @g1204 !

msorvig commented 1 year ago

However also note that -s EXPORTED_FUNCTIONS is not supported for -MAIN_MODULE=1:

em++: warning: EXPORTED_FUNCTIONS is not valid with LINKABLE set (normally due to SIDE_MODULE=1/MAIN_MODULE=1) since all functions are exported this mode.

MAIN_MODULE=2 can be used, but that mode has other effects which might not be wanted.

msorvig commented 3 months ago

This should help/fix the issue: https://github.com/emscripten-core/emscripten/pull/22095

emscripten-core / emscripten

Dynamic linking emscripten::val(..) fails in SIDE_MODULE #16711