search

emscripten-core / emscripten

Emscripten: An LLVM-to-WebAssembly Compiler
Other
25.64k stars 3.29k forks source link

Opening a side module larger than 4KB fails on Chrome #11753

Open cuinjune opened 4 years ago

cuinjune commented 4 years ago

Hi, I'm trying to dynamically load a side module from the main module using dlopen(). The side module loads fine as long as its size is smaller than 4KB but I need to load large-size side modules. Here's a simple code you can test this:

side.c:

#define SIZE 4000
char dummy[SIZE] = {};

int side(int a)
{
    return SIZE;
}

main.c:

#include <stdio.h>
#include <dlfcn.h>

int main() 
{
    void *handle;
    typedef int (*func_t)(int);

    handle = dlopen("side.wasm", RTLD_NOW);

    if (!handle) {
        printf("failed to open the library\n");
        return 0;
    }
    func_t func = (func_t)dlsym(handle, "side");

    if (!func) {
        printf("failed to find the method\n");
        dlclose(handle);
        return 0;
    }
    printf("side module size: %d byte\n", func(1));
}

index.html:

<!DOCTYPE html>
<html>

<head>
</head>

<body>
  <script async src="main.js"></script>
</body>

</html>

commands:

emcc side.c -s SIDE_MODULE=1 -o side.wasm
emcc main.c -s MAIN_MODULE=1 -o main.html --preload-file side.wasm
python3 -m http.server 8080

And this is the result I get in the Chrome browser:

Error in loading dynamic library side.wasm: RangeError: WebAssembly.Compile is disallowed on the main thread, if the buffer size is larger than 4KB. Use WebAssembly.compile, or compile on a worker thread.

Can someone please guild me on how to dynamically load a side module larger than 4KB? Thank you in advance!

sbc100 commented 1 year ago

@pd-l2ork, it seems likely that the change that broke your solution was #19310. This change moved the filesystem reading part of library loading into native code which is now part of dlopen. The behaviour of loadDynamicLibrary was also changed, on the assumption that this was an internal function with the public interface being dlopen. My advice would be to switch to using dlopen directly, unless there is some fundamental reason why that doesn't work for for you?

By the way the lines in your binary such as {"filename":"/pd-l2ork/extra/maxlib/average.wasm","start":15274628,"end":15276992} are the result of the --preload-file flag.

A few other (unrelated) notes on your command line flags.

pd-l2ork commented 1 year ago

Thank you @sbc100 for your reply. I think the original issue is that many of these objects are larger than 4KB (see the thread title). Has this been resolved? If so, would you be able to provide a skeleton example function that does the same replacing the __PD_loadlib one? Thank you again for your help.

sbc100 commented 1 year ago

Chrome has never supported loading module >4kb synchronously. The only way I know of that works around that would be via preloading of the modules, which --preload-file and --embed-file should both take care of.

See preloadedWasm in library_dylink.js for how this work.

pd-l2ork commented 1 year ago

Thanks again @sbc100 . So, now I tried changing code to use dlopen (I tried transplanting the original code by @cuinjune ) and now the system is unable to locate the file. Is dlopen trying to search --preload-file database/list (or whatever it is), or is it trying to locate the file on the local drive where the server is running? If it is going through the list, I have objects that share the same name (but are in different subfolders). How could one distinguish between the two, when the code above shows only referencing the actual filename.wasm? In other words, if, for instance the stored object is:

{"filename":"/pd-l2ork/extra/maxlib/average.wasm","start":15274628,"end":15276992}

should the file to be loaded be referenced as:

And what is this location in reference to? To the folder where the server was originally started?

pd-l2ork commented 1 year ago

In follow-up to my previous question, how would dlopen example code look inside js? When I try to compile js code with dlopen function, it says it does not recognize the function. Here's the original JS code using loadDynamicLibrary:

EM_JS(int, __Pd_loadLib, (const char *filename, const char *symname), {
    return Asyncify.handleAsync(async () => {
        try {
            await loadDynamicLibrary(UTF8ToString(filename), {loadAsync: true, global: true, nodelete: true, fs: FS});
            var makeout = Module['_' + UTF8ToString(symname)];
            if (typeof makeout === "function") {
                makeout();
                console.log("...success");
                return 1; // success
            }
            else {
                console.log("...no function found");
                return -1; // couldn't find the function
            }
        }
        catch (error) {
            console.log("...failed");
            console.error(error);
            return 0; // couldn't load the external
        }
    });
});

The C version of dlopen is seen in the top post in this thread:

    void *handle;
    typedef int (*func_t)(int);

    handle = dlopen("side.wasm", RTLD_NOW);

    if (!handle) {
        printf("failed to open the library\n");
        return 0;
    }
    func_t func = (func_t)dlsym(handle, "side");

    if (!func) {
        printf("failed to find the method\n");
        dlclose(handle);
        return 0;
    }
    printf("side module size: %d byte\n", func(1));

Note this is a part of the main, so I am not even sure how this would look as a separate C function, since I am quite sure the handle would disappear at the end of that function call.

So, how would this dlopen code look inside a js call, like the one above that is wrapped into EM_JS?

sbc100 commented 1 year ago

I believe with pd-l2ork/extra/maxlib/average.wasm or /pd-l2ork/extra/maxlib/average.wasm should work. In the past did average.wasm previously work? I would be surprised if that was the case.

dlopen is a native call so if you don't need to use an EM_JS function at all to call it. You can, for example, just write a function that calls dlopen + dlsym and then returns that function pointer (i.e. the result of dlsym).

kleisauke commented 1 year ago

FWIW, Chrome extended this limit to 8MB in version 115, see commit https://github.com/chromium/chromium/commit/d1a1a8fdd9b28feaf5d3accc0578092444a1083e.

pd-l2ork commented 1 year ago

Thank you for the clarification. After further hacking I am now finding that the object is successfully loaded. However, for some reason, the core code that this project is trying to port into a browser environment, seems to continue iterating looking for the object, even though it has supposedly found it. I suspect this is because it needs the pointer...

Also, to clarify, average.wasm was never used and it did not work when I manually entered it, so that part should be ok. More soon...

Thanks, again, for all your help.

pd-l2ork commented 1 year ago

@sbc100 @kripken OK, so I was able to verify that the new object is indeed loading the setup function. However, once the setup is loaded, the next is instantiation (average_new function located inside a separate .c file dedicated to this object/plugin that is to be dynamically loaded). What I found out is that if this function is declared as static, I end up having the Uncaught RuntimeError: null function or function signature mismatch error. If I recompile the object with that function not being static, everything works. This is a problem, because there are over a thousand of 3rd party objects, some that may share functions that are named the same, so it would be a really bad idea to keep all those functions in a global space. More so, the same source is used to compiile a native version of the software, which makes an additional argument against removing static declaration of those variables. So, how does one ensure that you can dlopen external object/library/plugin that may have its own static variables that should be exposed within the browser?

sbc100 commented 1 year ago

As far as I know there is no way to access at static C function from the DLL. static functions are internal to the object file and are not part of the DLL/shared library symbol table. I believe this is true on standard desktop systems such as linux, macOS and windows too. Are you saying you are able to use dlopen/dlsym to load static functions in some other platform?

pd-l2ork commented 1 year ago

This code is actively maintained on desktop and runs on Windows, Linux, and OSX. The average_new function being declared static is no problem because (I think--I am not the original author of this architecture) of the following:

1) average_setup function builds a list of methods (or functions) object has, and stores them in a class declaration (this is c code, so class is in reality a structure), that can be then referenced later when we may need to instantiate the same object (one or more times). In other words, the setup function takes care of the setting up of the class, and only after that is done is the object instantiated.

2) its instantiation function is invoked through that method, which is a pointer to the static function

As I mentioned, this works fine on all desktop platforms, so I am unsure why it is not working here.

The same code has also worked just fine on the previous version of emscripten that relied upon loadDynamicLibrary.

sbc100 commented 1 year ago

So IIUC there is some kind of setup function that returns at struct containing function pointers? That should work fine. Are you able to run the setup function? What does it return? Perhaps if you should share an example of the exact code that is failing that might help (in as simple a form as possible, if you can).

pd-l2ork commented 1 year ago

The setup function runs fine. I know this because when run it prints out a message. The average_new is never reached (I have another printout message on top of it). I wish I could simplify this to make it more easy to share. Alas, this may take me a couple of days just to come up with such an abridged version. The overall code base of this thing is humongous--git suggests over 1.5M lines of code, although some of this is definitely due to a collection of images.

sbc100 commented 1 year ago

How is average_new supposed to be called? Is it called via dlsym? If not, then how/when is it called? Is it just a the single setup function looked up via dlsym or are there other?

pd-l2ork commented 1 year ago

Here's a general breakdown:

The program (pd-l2ork) reads a text file (a.k.a. patch) that contains visual dataflow code split into visual object creation (think of it as boxes with names, each with particular function), object connection (visual cords connecting those boxes), and encapsulation (boxes grouped into other subpatches). Here, we are only concerned with object creation.

Note that everything I am about to share works perfectly in desktop environments and has worked fine in emscripten release sometime in 2021 using loadDynamicLibrary approach that @cuinjune implemented, as shown in the thread above.

Below are key structures and functions:

Core:

Core or external objects:

void average_setup(void)
{
   // this creates the class, passing it creation function, freeing function, and optional arguments
    average_class = class_new(gensym("average"), (t_newmethod)average_new,
        (t_method)average_free, sizeof(t_average), 0, A_DEFFLOAT, 0);
   // I added this to post the function address
    post("average_setup %lx", (t_method)average_new);
  // this adds additional methods, all of which are statically defined
    class_addmethod(average_class, (t_method)average_reset, gensym("reset"), 0);
    class_addmethod(average_class, (t_method)average_linear, gensym("linear"), 0);
    class_addmethod(average_class, (t_method)average_geometric, gensym("geometric"), 0);
    class_addmethod(average_class, (t_method)average_weight, gensym("weight"), 0);
    class_addfloat(average_class, average_float);
    class_addmethod(average_class, (t_method)average_index, gensym("index"), A_FLOAT, 0);

   // version is a const char text that is visible when I run my code, so the code gets this far
    logpost(NULL, 4, version);
}
static int sys_do_load_lib(t_canvas *canvas, const char *objectname,
    const char *path)
{
 // cutting bunch of irrelevant stuff
 // ...
// this is what it used to be and what used to call EM_JS __Pd_loadLib call that relied on the loadDynamicLibrary
// now it is commented out
/*
#ifdef __EMSCRIPTEN__
    post("__Pd_loadLib <%s> <%s>", filename, symname);
    int res = __Pd_loadLib(filename, symname);
    if (ret == 1) {
        post("...success");
        (*makeout)();
        class_set_extern_dir(&s_);
        return (1);
    }
    else if (ret == 0) {
        verbose(1, "%s: couldn't load", filename);
        post("...couldn't load");
        class_set_extern_dir(&s_);
        return (0);
    }
    else { //ret is -1, object was opened but no setup function was found
        post("...no setup function found");
        return(0);
    }
#elif _WIN32
*/
#ifdef _WIN32
   // this part is cut, as it does not apply
#elif defined(HAVE_LIBDL) || defined(__FreeBSD__) || defined(__EMSCRIPTEN__) // this is where I've now added EMSCRIPTEN part
   // this is what we use currently
    post("sys_do_load_lib <%s> <%s>", filename, symname);
    dlobj = dlopen(filename, RTLD_NOW | RTLD_GLOBAL);
    if (!dlobj)
    {
        error("%s: %s", filename, dlerror());
        class_set_extern_dir(&s_);
        return (0);
    }
    makeout = (t_xxx)dlsym(dlobj,  symname);
    if(!makeout)
        makeout = (t_xxx)dlsym(dlobj,  "setup");
#else
#warning "No dynamic loading mechanism specified, \
    libdl or WIN32 required for loading externals!"
#endif

    if (!makeout)
    {
        error("load_object: Symbol \"%s\" not found", symname);
        class_set_extern_dir(&s_);
        return 0;
    }
    (*makeout)();
    class_set_extern_dir(&s_);
    post("...sys_do_load_lib success");
    return (1);
}

When creating objects that come built-in with the core pd-l2ork package (unlike 3rd party objects), everything is created just fine. However, when trying to create dynamically loaded object, this is something that used to work fine around 2021 and now does not, with only notable change between now and then being emscripten version update. So, I set out to adapt loading using dlopen and here's the current status:

Core pd-l2ork reads the object creation request from the patch file. It searches the existing class list and if it does not find it, tries to look for an external. This eventually triggers sys_do_load_lib, which in turn triggers average_setup, which triggers class_new, and then invokes creation of the actual object.

The resulting relevant console printout excerpt shows what worked and where it fails:

// this is the call when pd-l2ork does not know what to instantiate, so it calls this to trigger search
// (1 stands for a number of arguments, in this case it is 10. this means that the object will be
// giving a running average over the last 10 numbers
main.js:298 new_anything <maxlib/average> 1
// here you see the call with proper location and setup function
main.js:298 sys_do_load_lib <pd-l2ork-web/extra/maxlib/average.wasm> <average_setup>
// here we see the address of the new method which is 2674
main.js:298 class_new newmethod 2674
// here we see methods being added (we have both the long and short version here,
// so both versions work)
main.js:298 addmethod total=297 <average>
main.js:298 addmethod total=298 <maxlib/average>
// here we can see that class_new newmethod has indeed correct address,
// matching the one above
main.js:298 class:average newmethod=2674
// here we see what the address of the newmethod is coming from the average_setup
// this also proves that average_setup is found and is triggered
main.js:298 average_setup 2674
// here you see how other methods (static functions inside average.c file) are being added,
// so that they can be referenced, so this tells us class has indeed gotten new methods
main.js:298 addmethod total=1 <reset>
main.js:298 addmethod total=2 <linear>
main.js:298 addmethod total=3 <geometric>
main.js:298 addmethod total=4 <weight>
main.js:298 addmethod total=5 <index>
// this one is also called from average_setup via logpost (level 4 in terms of verbosity)
main.js:298 verbose(4): average v0.2, written by Olaf Matthes <olaf.matthes@gmx.de>,
revised by Ivica Ico Bukvic <ico@vt.edu>
// this tells us that sys_do_load_lib has successfully completed
main.js:298 ...sys_do_load_lib success
// now that the class has been properly created, we call new_anything again
// this time it should be able to create an object because it has its class in its class list
main.js:298 ...new_anything typedmess <maxlib/average> 1
// unfortunately, this is what happens if average_new is a static function (typedmess is
// simply resending the patch text asking to instantiate the object)
// if I make average not static, the object instantiates fine but that is not an option,
// as doing so will very likely create address clashes among over a thousand of 3rd party objects
main.wasm:0x1359d1 Uncaught (in promise) RuntimeError: null function or function
signature mismatch
    at main.wasm:0x1359d1
    at main.wasm:0x135344
    at main.wasm:0x13581d
    at main.wasm:0x130275
    at main.wasm:0x12169d
    at main.wasm:0x12155d
    at main.wasm:0x13557b
    at main.wasm:0x130275
    at main.wasm:0x133a73
    at main.wasm:0x133b0e
$pd_typedmess @ main.wasm:0x1359d1
$new_anything @ main.wasm:0x135344
$pd_typedmess @ main.wasm:0x13581d
$binbuf_eval @ main.wasm:0x130275
$func1654 @ main.wasm:0x12169d
$canvas_obj @ main.wasm:0x12155d
$pd_typedmess @ main.wasm:0x13557b
$binbuf_eval @ main.wasm:0x130275
$binbuf_evalfile @ main.wasm:0x133a73
$glob_evalfile @ main.wasm:0x133b0e
$libpd_openfile @ main.wasm:0x187a73
$_ZN2Pd9openPatchERKNSt3__212basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEES8_ @ main.wasm:0xaa48b
$_ZN10emscripten8internal13MethodInvokerIM2PdFvRKNSt3__212basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEESB_EvPS2_JSB_SB_EE6invokeERKSD_SE_PNS0_11BindingTypeIS9_vEUt_ESL_ @ main.wasm:0xa920a
Pd$openPatch @ VM121:10
openPatch @ main.js:2094
init @ main.js:2153
await in init (async)
mainInit @ main.js:840
__mainInit @ main.js:1
$__original_main @ main.wasm:0xa8817
$main @ main.wasm:0xa9488
callMain @ main.js:1
doRun @ main.js:1
run @ main.js:1
runCaller @ main.js:1
removeRunDependency @ main.js:1
processPackageData @ main.js:1
(anonymous) @ main.js:1
xhr.onload @ main.js:1
load (async)
fetchRemotePackage @ main.js:1
loadPackage @ main.js:1
(anonymous) @ main.js:1
(anonymous) @ main.js:1

Below is the relevant build output:

average.c:

emcc -DPD -I/home/l2orkist/Downloads/pd-l2ork/emscripten/../pd/src -Wall -W -Wno-unused-parameter -s SIDE_MODULE=1 -O3 -o "average.o" -c "average.c"
average.c:218:22: warning: format string is not a string literal (potentially insecure) [-Wformat-security]
  218 |     logpost(NULL, 4, version);
      |                      ^~~~~~~
average.c:218:22: note: treat the string as an argument to avoid this
  218 |     logpost(NULL, 4, version);
      |                      ^
      |                      "%s", 
1 warning generated.
emcc -s SIDE_MODULE=1 -O3 -o "average.wasm" "average.o"   
chmod a-x "average.wasm"

main (combination of core and all bundled externals):

emcc -I/home/l2orkist/Downloads/pd-l2ork/emscripten/build/../../pd/src -I/home/l2orkist/Downloads/pd-l2ork/emscripten/build/../../libpd/libpd_wrapper -O3 -s MAIN_MODULE=1 --bind -o main.html /home/l2orkist/Downloads/pd-l2ork/emscripten/build/../../emscripten/src/main.cpp /home/l2orkist/Downloads/pd-l2ork/emscripten/build/../../emscripten/src/pd.cpp /home/l2orkist/Downloads/pd-l2ork/emscripten/build/../../emscripten/src/js.cpp \
-s USE_SDL=2 -s ERROR_ON_UNDEFINED_SYMBOLS=0 -s ALLOW_MEMORY_GROWTH=1 \
-s FORCE_FILESYSTEM=1 -s EXPORTED_RUNTIME_METHODS=FS \
--preload-file pd-l2ork-web -L/home/l2orkist/Downloads/pd-l2ork/emscripten/build/../../libpd/libs -lpd -lm
sbc100 commented 1 year ago

Regarding the old EMJS code, I think I do see a problem there. The code is expecting symbols from the loaded library to be presend on the global Module object with the underscore prefix. i.e. `var makeout = Module['' + UTF8ToString(symname)];`

These days we use wasmImports as the global symbol table. So that line should be replaced with var makeout = wasmImports[UTF8ToString(symname)];. That is, assuming that loadDynamicLibrary succeeds.

However, the dlopen approach should also work and be simpler and more robust over time. Are the dlsym calls succeeding? Presumable one of the gensym calls is failing? What does gensym do exactly? Does it call dlsym?

sbc100 commented 1 year ago

If you want to get better stack traces can you try building with -g (or just --profiling-funcs). That should give function names rather than just:

signature mismatch
    at main.wasm:0x1359d1
    at main.wasm:0x135344
    at main.wasm:0x13581d
    at main.wasm:0x130275
    at main.wasm:0x12169d
    at main.wasm:0x12155d
    at main.wasm:0x13557b
    at main.wasm:0x130275
    at main.wasm:0x133a73
    at main.wasm:0x133b0e
pd-l2ork commented 1 year ago

These days we use wasmImports as the global symbol table. So that line should be replaced with var makeout = wasmImports[UTF8ToString(symname)];. That is, assuming that loadDynamicLibrary succeeds.

Thank you. I will try to revert back to that implementation and see how it goes.

However, the dlopen approach should also work and be simpler and more robust over time. Are the dlsym calls succeeding? Presumable one of the gensym calls is failing? What does gensym do exactly? Does it call dlsym?

gensym is simply generating a symbol (string) that is kept in a string table. This is needed for a number of reasons, including wirelessly connecting objects via internal send and receive objects (among other things).

BTW, dlopen does work up through the setup function. Where it fails is when trying to instantiate average_new, which should've been stored in class_new, and whose address, per my debug printouts, appears to be tracking across multiple function calls.

pd-l2ork commented 1 year ago

If you want to get better stack traces can you try building with -g (or just --profiling-funcs). That should give function names rather than just:

signature mismatch
    at main.wasm:0x1359d1
    at main.wasm:0x135344
    at main.wasm:0x13581d
    at main.wasm:0x130275
    at main.wasm:0x12169d
    at main.wasm:0x12155d
    at main.wasm:0x13557b
    at main.wasm:0x130275
    at main.wasm:0x133a73
    at main.wasm:0x133b0e

I will also make sure to compile with a debug flag and will let you know. Thank you.

pd-l2ork commented 1 year ago

So, I am completely baffled. I just went to enable the debug options and I don't think I even did it on all components that comprise the final main wasm file and upon recompiling dlopen worked as expected. I then, went on to disable debugging to revert everything back and was unable to break it again. Eventually, I was able to enable debugging for everything and that ended-up spawning some odd errors, but this is a separate issue at this point. I know for a fact that I cleaned the entire tree before. Could it be that there may have been a stale .o file that could've caused this? I am thinking .o file from a different compile time may generate a different signature, resulting in an incompatible build perhaps? I am at a loss, and my brain is fried... More tomorrow... Thanks again for all your help.

pd-l2ork commented 1 year ago

For whatever rason I am now unable to break it in the way it was broken before, but am now running into the debug-enabled version throwing bunch of errors that are not present in an optimized build. Below is an example of the output. Everything loads fine in the browser, but as son as I click, I get the following stream of errors (they repeat, so below is only a snapshot). None of this is a problem when running -O3 build (this one is -g):

main.js:56 Aborted(Assertion failed: exceptfds not supported)
printErr @ main.js:56
abort @ main.js:950
assert @ main.js:631
___syscall__newselect @ main.js:5420
$select @ main.wasm:0x28f3a7
$sys_domicrosleep @ main.wasm:0x15ce2e
$sys_pollgui @ main.wasm:0x15eb39
$libpd_process_float @ main.wasm:0x1aa664
$Pd::audioOut(unsigned char*, int) @ main.wasm:0xb90db
$Pd::forwardAudioOut(void*, unsigned char*, int) @ main.wasm:0xb6c3b
$HandleAudioProcess @ main.wasm:0x209500
dynCall @ main.js:1622
SDL2.audio.scriptProcessorNode.onaudioprocess @ main.js:1287
main.js:969 Uncaught RuntimeError: Aborted(Assertion failed: exceptfds not supported)
    at abort (main.js:969:11)
    at assert (main.js:631:5)
    at ___syscall__newselect (main.js:5420:7)
    at select (main.wasm:0x28f3a7)
    at sys_domicrosleep (main.wasm:0x15ce2e)
    at sys_pollgui (main.wasm:0x15eb39)
    at libpd_process_float (main.wasm:0x1aa664)
    at Pd::audioOut(unsigned char*, int) (main.wasm:0xb90db)
    at Pd::forwardAudioOut(void*, unsigned char*, int) (main.wasm:0xb6c3b)
    at HandleAudioProcess (main.wasm:0x209500)
abort @ main.js:969
assert @ main.js:631
___syscall__newselect @ main.js:5420
$select @ main.wasm:0x28f3a7
$sys_domicrosleep @ main.wasm:0x15ce2e
$sys_pollgui @ main.wasm:0x15eb39
$libpd_process_float @ main.wasm:0x1aa664
$Pd::audioOut(unsigned char*, int) @ main.wasm:0xb90db
$Pd::forwardAudioOut(void*, unsigned char*, int) @ main.wasm:0xb6c3b
$HandleAudioProcess @ main.wasm:0x209500
dynCall @ main.js:1622
SDL2.audio.scriptProcessorNode.onaudioprocess @ main.js:1287
main.js:56 Aborted(Assertion failed: exceptfds not supported)
printErr @ main.js:56
abort @ main.js:950
assert @ main.js:631
___syscall__newselect @ main.js:5420
$select @ main.wasm:0x28f3a7
$sys_domicrosleep @ main.wasm:0x15ce2e
$sys_pollgui @ main.wasm:0x15eb39
$libpd_process_float @ main.wasm:0x1aa664
$Pd::audioOut(unsigned char*, int) @ main.wasm:0xb90db
$Pd::forwardAudioOut(void*, unsigned char*, int) @ main.wasm:0xb6c3b
$HandleAudioProcess @ main.wasm:0x209500
dynCall @ main.js:1622
SDL2.audio.scriptProcessorNode.onaudioprocess @ main.js:1287
main.js:969 Uncaught RuntimeError: Aborted(Assertion failed: exceptfds not supported)
    at abort (main.js:969:11)
    at assert (main.js:631:5)
    at ___syscall__newselect (main.js:5420:7)
    at select (main.wasm:0x28f3a7)
    at sys_domicrosleep (main.wasm:0x15ce2e)
    at sys_pollgui (main.wasm:0x15eb39)
    at libpd_process_float (main.wasm:0x1aa664)
    at Pd::audioOut(unsigned char*, int) (main.wasm:0xb90db)
    at Pd::forwardAudioOut(void*, unsigned char*, int) (main.wasm:0xb6c3b)
    at HandleAudioProcess (main.wasm:0x209500)
abort @ main.js:969
assert @ main.js:631

I tried compiling with -s USE_PTHREADS as one of the online resources suggested, but that made things even worse, making the program fail to even load (threw one error that I don't have on hand and stopped right there).

Should I make this a separate issue?

pd-l2ork commented 1 year ago

Here is a more on-topic question. So, when I run a patch (a snippet of code) that does not require dlopen, everything works fine. However, if I have a patch that invokes dlopen AND the object in question is supposed to process audio, even though the object now loads correctly, as soon as I do anything on the page with my mouse (e.g. click onto a visual widget, a slider or a button), I get the following error:

main.wasm:0x1342f3 Uncaught RuntimeError: memory access out of bounds
    at main.wasm:0x1342f3
    at main.wasm:0x188d52
    at main.wasm:0xaa597
    at main.wasm:0xa9335
    at ClassHandle.Pd$sendFloat [as sendFloat] (eval at newFunc (main.js:1:396680), <anonymous>:10:1)
    at gui_slider_bang (main.js:1407:15)
    at gui_slider_onmousemove (main.js:1440:9)
    at main.js:1615:9

The source of error may be different depending on what I pressed, but the end result is always the same. I tried recompiling with -s TOTAL_MEMORY=128MB instead of -s ALLOW_MEMORY_GROWTH=1 thinking perhaps resizing the heap may screw up addresses which are stored in the class, in case the entire thing is repositioned to a different memory address, but that made no difference.

sbc100 commented 1 year ago

Looks like a memory corruption issue. I would recommend a debug build with -sASSERTIONS and --profiling-funcs. That might. You might also want to try -fsanitize=address. It sounds like we are getting off topic for this issue though.

sbc100 commented 1 year ago

Regarding the original issue: Chome just increased the limit from 4kb to 8Mb https://github.com/chromium/chromium/commit/d1a1a8fdd9b28feaf5d3accc0578092444a1083e. So this issue should go away once that hits chrome stable.

pd-l2ork commented 1 year ago

So, as a closure to this discussion, I learned two critical things:

1) fixed exceptfds issue by updating the source (see #19951 for more info) 2) learned that -fsanitize=address is the magical thing that solves all my problems. If I don't use this compile flag, I get random out of memory errors that happen all over the code, which is unusual for a fairly mature source base that has been running on other platforms for years. Once I enabled this flag, both -g and -O3 builds run just fine. I imagine perhaps this flag makes address handling stricter, which is essential for the way m_class functions, as described above. Is this correct? Or, could this be an emscripten bug of sorts? In other words, why not make this default?

Thanks @sbc100 again for all your time and assistance.