Using WasmFS / OPFS from the main thread deadlocks on Safari

[fguinan]

Please include the following in your bug report:

Version of emscripten/emsdk: emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 3.1.42 (6ede0b8fc1c979bb206148804bfb48b472ccc3da) clang version 17.0.0 (https://github.com/llvm/llvm-project f3b64887de61020c09404bfee97b2fadd30df10a) Target: wasm32-unknown-emscripten Thread model: posix InstalledDir: <snip>

Full link command and output with -v appended: emcc main.cpp -o hello.html -sWASMFS -pthread -sPTHREAD_POOL_SIZE=4

#include <emscripten.h>
#include <emscripten/wasmfs.h>
#include <unistd.h>

#include <filesystem>
#include <string>

#include <thread>

extern "C"
{
    std::thread* bgThread = nullptr;

    EMSCRIPTEN_KEEPALIVE int main() 
    {
        // Initialize wasmfs in bg thread
        bgThread = new std::thread([]() 
        {
            auto wasmfs_backend = wasmfs_create_opfs_backend();
            const char* rootMount = "/persistent";
            bool exists = access(rootMount, F_OK) == 0;
            if (!exists)
                wasmfs_create_directory(rootMount, 0777, wasmfs_backend);
        });

         // Keep the standard library runtime alive even after exiting main`
        emscripten_exit_with_live_runtime();
        return 0;
    }

    EMSCRIPTEN_KEEPALIVE void checkForDirectory()
    {
        bool exists = std::filesystem::exists("/persistent/data");
        printf("exists = %d\n", exists);
    }
}

Load the html with the necessary COOP settings (in order to get pthread to work). Then in the web debugger console, make a call to window.Module._checkForDirectory(). In Chrome, the output is printed out successfully. In Safari, the output is never printed out. The main thread is stuck in a tight loop, and the debugger is no longer useable. If there were other web workers running, their execution also seems to pause.

[sbc100]

Is this a regression? (i.e. has this ever worked for you?)

[fguinan]

We ported some code to web, and we focused our initial testing on Chrome. We just started trying to validate on Safari, which is when we discovered this issue. So this is not a regression from our point of view, but I can't say whether it's a regression in general in emscripten or Safari.

[fguinan]

Just wondering if there has been any insight gained on this issue yet? Thanks.

cc: @tlively, @sbc100

[tlively]

Hi @fguinan, sorry, we've started looking into this but we don't know what's going on yet.

[brendandahl]

This appears to be something buggy happening in Safari's file system API. Safari gets stuck when calling getFileHandle on the non existent file and never resolves or rejects the promise when using await. If I don't await the promise and instead do something like this:

      parentHandle.getFileHandle(name, {create: create}).then(
        (value) => {
          console.log("found " + value);
        }, (error) => {
          console.log("rejected"); console.log(error);
        });
       return -{{{ cDefs.EEXIST }}};

Then I will see the rejection in the console as expected.

So far my attempt at creating a minimal JS reproducer has not worked.

[fguinan]

Happy new year! Curious if there was any more progress on this issue. Thanks.

[fguinan]

Playing around with this a bit, it seems as if getFileHandle() requires cycles on the main browser thread when called on a non-existent file. Only if the main browser thread is free will the promise actually reject. However, if the main browser thread is tied up, the promise won't be rejected (until the main browser thread becomes free).

Interestingly enough, if the file is actually a directory, then getFileHandle() will reject properly with a "TypeMismatchError". It is only for "NotFoundError" errors that then the main browser thread is required.

@tlively, @brendandahl - Is this observation consistent with what you are seeing? Any thoughts on a workaround? E.g. are there any other browser APIs for OPFS that can determine whether a file exists? Should a bug be logged with Apple or WebKit on this issue?

[brendandahl]

I tried a bit more to make a smaller test case, but didn't have any luck. I think it would be worthwhile to file a bug with WebKit and see if they have any insights. If we don't hear anything from them we can then try and look into workarounds.

[fguinan]

It really is looking to me that the JS APIs called on the proxy thread require cycles on the main browser thread. The following sample code sleeps the main thread for 10 seconds while calling std::filesystem::exists() from a background thread. The background thread does not finish until the main thread finishes its sleep cycle.

I also noticed the same issue with std::fstream.

We're going to try moving our code off the main browser thread to avoid this issue. We now believe synchronous file I/O from the main browser thread is just not supported with WASMFS / OPFS in Safari (and Firefox).

#include <emscripten.h>
#include <emscripten/console.h>
#include <emscripten/wasmfs.h>
#include <unistd.h>

#include <filesystem>
#include <fstream>
#include <future>
#include <string>

#include <thread>

extern "C"
{
    std::thread* bgThread = nullptr;

    EMSCRIPTEN_KEEPALIVE int main() 
    {
        // Initialize wasmfs in bg thread
        bgThread = new std::thread([]() 
        {
            auto wasmfs_backend = wasmfs_create_opfs_backend();
            const char* rootMount = "/persistent";
            bool exists = access(rootMount, F_OK) == 0;
            if (!exists)
                wasmfs_create_directory(rootMount, 0777, wasmfs_backend);

            std::filesystem::create_directories("/persistent/data");
        });

         // Keep the standard library runtime alive even after exiting main
        emscripten_exit_with_live_runtime();
        return 0;
    }

    EMSCRIPTEN_KEEPALIVE void checkForDirectory()
    {
        // bool exists = std::filesystem::exists("/persistent/data");
        // printf("exists = %d\n", exists);

        bool exists = false;
        printf("exists = %d\n", exists);
        std::promise<bool> resultPromise;
        emscripten_console_error("on main thread, about to spawn thread");
        bgThread = new std::thread([&exists, &resultPromise]() 
        {
            emscripten_console_error("on bg thread, calling exists()");
            exists = std::filesystem::exists("/persistent/data1");
            // printf("exists = %d\n", exists1);
            emscripten_console_error("on bg thread, after calling exists()");

            resultPromise.set_value(true);
        });

        using namespace std::chrono_literals;
        std::this_thread::sleep_for(10000ms);

        // std::future<bool> resultFuture = resultPromise.get_future();
        // emscripten_console_error("on main thread, about to wait on future");
        // exists = resultFuture.get();
        // emscripten_console_error("on main thread, after getting future");

        // bgThread->join();
        // delete bgThread;
        // printf("exists = %d\n", exists);
    }
}

[brendandahl]

One thing that my be confusing the results is how the console appears to work in safari. It seems logs from workers can't be processed while the main thread is busy. For example:

#include <emscripten.h>
#include <emscripten/console.h>
#include <emscripten/html5.h>
#include <unistd.h>
#include <string>
#include <thread>

extern "C"
{
    std::thread* bgThread = nullptr;

    EMSCRIPTEN_KEEPALIVE int main() 
    {
        // Keep the standard library runtime alive even after exiting main
        emscripten_exit_with_live_runtime();
        return 0;
    }

    EMSCRIPTEN_KEEPALIVE void checkForDirectory()
    {
        emscripten_console_logf("on main thread, about to spawn thread %.0f\n", emscripten_date_now() / 1000);
        bgThread = new std::thread([]() 
        {
            emscripten_console_logf("on bg thread %.0f\n", emscripten_date_now() / 1000);
        });

        using namespace std::chrono_literals;
        std::this_thread::sleep_for(10000ms);
        emscripten_console_logf("done %.0f\n", emscripten_date_now() / 1000);
    }
}

Chrome

Module._checkForDirectory()
hello.js:2149 on main thread, about to spawn thread 1706299938
hello.js:2149 on bg thread 1706299938
hello.js:2149 done 1706299948

Safari

[Log] on main thread, about to spawn thread 1706300004 (hello.js, line 2149)
[Log] done 1706300014 (hello.js, line 2149)
[Log] on bg thread 1706300004 (hello.js, line 2149)

On safari it looks like the BG thread doesn't run until the end based on the output order, but if you look at the timestamp it actually did run earlier at the same time as the first log.

[fguinan]

Ah yes, thank you, I struggled with trying to get the logs to show what I was seeing. If I add timestamps to my last example, I still see the bg thread blocked until the main thread is free. (This only occurs when the file does not exist. However, it also happens when attempting to use std::fstream APIs on existing files.)

Chrome

exists = 0
hello.js:2539 on main thread, about to spawn thread at time 1706303820
hello.js:2539 on bg thread, calling exists() at time 1706303820
hello.js:2539 on bg thread, after calling exists() at time 1706303820
hello:1237 exists = 0

Safari

[Log] exists = 0 (hello, line 1237)
[Log] on main thread, about to spawn thread at time 1706303609 (hello.js, line 2539)
[Log] on bg thread, calling exists() at time 1706303609 (hello.js, line 2539)
[Log] on bg thread, after calling exists() at time 1706303619 (hello.js, line 2539)
[Log] exists = 0 (hello, line 1237)

[thearperson]

Just bumped into the same issue here.

[patrickcorrigan]

Can also confirm that this hangs in Safari 18 but runs fine in Chrome. Any updates or ideas on this?

#include <pthread.h>
#include <emscripten.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdio.h>
#include <assert.h>
#include <string.h>
#include <emscripten/wasmfs.h>

// Function that runs in a new thread to mount OPFS and perform file operations
void* mount_opfs(void* arg) {
    // Create the OPFS backend
    backend_t opfs = wasmfs_create_opfs_backend();
    assert(opfs);

    // Mount the OPFS backend to /opfs
    int err = wasmfs_create_directory("/opfs", 0777, opfs);
    assert(err == 0);
    printf("Mounted OPFS at /opfs\n");

    int fd = open("/opfs/test.txt", O_RDWR | O_CREAT | O_TRUNC, 0777);
    assert(fd > 0);
    const char* msg = "Hello from OPFS in a thread!";
    int nwritten = write(fd, msg, strlen(msg));
    assert(nwritten == strlen(msg));
    printf("Wrote to file: %s\n", msg);

    close(fd);

    return NULL;
}
int main() {
    pthread_t thread;
    int result = pthread_create(&thread, NULL, mount_opfs, NULL);
    assert(result == 0);
}

void read_opfs_file() {
    int fd = open("/opfs/test.txt", O_RDONLY);
    assert(fd > 0);

    char buffer[256];
    int nread = read(fd, buffer, sizeof(buffer) - 1);
    assert(nread >= 0);

    buffer[nread] = '\0';
    printf("Read from file: %s\n", buffer);

    close(fd);
}