emscripten-core / emscripten

Emscripten: An LLVM-to-WebAssembly Compiler
Other
25.69k stars 3.29k forks source link

Support Uint8Array instances as Module.stdin and as Module's stdout/stderr #13476

Open vadimkantorov opened 3 years ago

vadimkantorov commented 3 years ago

Overriding Module.stdin and doing an iterator works, but is very slow for large-ish inputs:

            stdin()
            {
                if(Module.input_stdin_binary.length == 0)
                    return null;
                const ord = Module.input_stdin_binary[0];
                Module.input_stdin_binary = Module.input_stdin_binary.slice(1);
                return ord;
            },

I'm using this to calculate sha1 hash with busybox's sha1sum. I'm comparing providing file contents as stdin and calling sha1sum versus saving the contents to a temp file and calling sha1sum. The first is so much slower - probably for no good reason

curiousdannii commented 3 years ago

For apps that use stdin a lot it would be nice to have a way to pass whole arrays/buffers in.

I made this work-around for my project, to intercept calls to getc and read from a buffer. https://github.com/curiousdannii/emglken/blob/master/src/getc.c

But if you need more functions it would get increasingly messy. And it requires ASYNCIFY.

vadimkantorov commented 3 years ago

Okay, this version is much faster, but the original feature request to allo direct uint8array consumption still stands:

stdin()
{
    const ord = Module.input_stdin_binary[Module.input_stdin_binary_iterator++];
    return ord === undefined ? null : ord;
}
curiousdannii commented 3 years ago

I don't know if this would be enough for your use case, but some people might only need to provide a single buffer at the start (in the Module object), which could be handled without too much difficulty. But if someone needed to provided more buffers later on, then that would be much more complex.

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because there has been no activity in the past year. It will be closed automatically if no further activity occurs in the next 30 days. Feel free to re-open at any time if this issue is still relevant.

vadimkantorov commented 2 years ago

stale bump

vadimkantorov commented 10 months ago

Same for stdout. It'd be much more convenient to have such an option as built-in instead of inefficiently appending bytes in stdout(ord) overload in JavaScript

vadimkantorov commented 10 months ago

It would also be nice to support bash-like redirects from/to file