Demo crashes from high memory usage

[guest271314]

The demo eventually crashes at Chromium 89

[JohnStarich]

Thanks for the report @guest271314! Briefly Googling the message Error code: 9, I'm seeing it could be any number of things, including excessively high memory usage. Unfortunately, Go Wasm needs more dev time put in for memory optimizations. Currently the entire file system is set up in memory, including the Go compiler's assets.

I think I hit the same error you did in my Linux VM. Once I gave the machine more RAM it worked on Chromium 89:

Can you confirm if giving the browser more memory fixes the issue? I know that's not ideal, but it would be good to confirm the cause.

[guest271314]

I am not sure what you mean by giving the browser more memory? I can run BleachBit to clean memory. If the user is running entirely on RAM it is probably not possible to add more RAM dynamically after startup.

I do not know Go and not completely sure how you have set this up, though at Chromium we can write directly to the local filesystem using File System Access API (Native File System). I have used inotify-tools https://github.com/guest271314/captureSystemAudio/blob/master/notify.sh at Chromium https://github.com/guest271314/captureSystemAudio#usage to run arbitrary code when files are opened, which could be substituted for pure memory usage here. The directory could then be deleted once the process is completed. I am mostly interested in how developers are running native applications and getting STDOUT. I have achieved that a few different ways and now trying to determine the simplest.

[JohnStarich]

Yeah, that's the idea. In my case I had to close Chromium's other windows to reclaim enough raw memory on the machine so it would launch. Fully understand the frustration there though. I have experimented with using IndexedDB as a way to move the file system onto disk, but I hit several issues I haven't been able to solve on my own. You're welcome to try it out, though. The IndexedDB adapter is still wired in, just not enabled by default (due to various hangs and crashes in the WASM implementation).

... though at Chromium we can write directly to the local filesystem using File System Access API (Native File System).

That sounds great! I'm not familiar with the API yet, but that could be a good solution. We may need to give IndexedDB another shot first, since it's more widely supported at the moment, but I've been keeping an eye on Chromium's newer file API. It looks promising for sure!

[JohnStarich]

If you're interested in hacking on go-wasm, I'd be happy to give you some pointers! Notably, the indexeddb package is the current experimental implementation. It's wired in using mountfs.

You can actually mount it from JS in the demo too, using window.overlayIndexedDB()

[guest271314]

At Chromium or Chrome File System Access API (formerly native File System) provides a means to write directly to local filesystem. I have not tried IndexedDB. I do not want the browser having any control over what I write, and AFAICT IndexedDB is under the influence of memory and hard disk quotas, similar to WebAssembly.Memory.grow() https://bugs.chromium.org/p/v8/issues/detail?id=7881 (see also https://github.com/guest271314/proposal-resizablearraybuffer) and Blob size restrictions that were recently adjusted.

On the path you on currently on relevant to session and local storage @fivedots have done and are doing with NativeIO, see https://github.com/fivedots/nativeio-explainer/issues/4, https://github.com/fivedots/nativeio-porting-tutorial.

I do not get frustrated. I create workarounds. Mission statement: Fix WontFix.

WebTransport https://github.com/GoogleChrome/samples/tree/gh-pages/webtransport provides a means to write to and read STDOUT https://github.com/guest271314/setUserMediaAudioSource. You can write to local filesystem, or do whatever you want.

If you're interested in hacking on go-wasm, I'd be happy to give you some pointers!

I am interested in reading and writing to and from STDIN and STDOUT. I do not have a use for IndexedDB or NativeIO or other browser-backed private origin (sandboxed) filesystems (we can get and change the data anyway if we really want to https://stackoverflow.com/a/36098618).

Yes. I am interested in learning. Where to begin? All I know about Python I learned over the past few weeks experimenting with the sample sample code from https://github.com/GoogleChrome/samples/blob/gh-pages/quictransport/quic_transport_server.py using the now obsolete QuickTransport.

[guest271314]

@JohnStarich What interests me in your work is that you compiled Go to WASM

Go Wasm is a Go development environment with the essentials to write and run code entirely within the browser, using the power of WebAssembly (Wasm).

One project that I have not completed is porting pavucontrol, just pactl and parec and parecord or the entire PulseAudio to JavaScript. I solved https://gitlab.freedesktop.org/pulseaudio/pavucontrol/-/issues/91, https://github.com/lxqt/pavucontrol-qt/issues/161 using the code at the previously linked repository. Ideally we will have the pactl and optionally the GUI for pavucontrol or the entire pulseaudio binary at our disposal in JavaScript. Another project is compiling Chromium Streams implementation to WASM, see https://github.com/MattiasBuelens/web-streams-polyfill/issues/20.

[JohnStarich]

At Chromium or Chrome File System Access API (formerly native File System) provides a means to write directly to local filesystem. I have not tried IndexedDB. I do not want the browser having any control over what I write, and AFAICT IndexedDB is under the influence of memory and hard disk quotas, similar to WebAssembly.Memory.grow() https://bugs.chromium.org/p/v8/issues/detail?id=7881 (see also https://github.com/guest271314/proposal-resizablearraybuffer) and Blob size restrictions that were recently adjusted.

Yes, great points. I've looked briefly at the Chrome File System Access API. My impression is it would be a great integration, but it looked experimental to me? If it looks like it will hit stable eventually, I think that's a good fit for go wasm 👍

The initial reason I started with IndexedDB is because it offered a rich API for large blobs of arbitrary data, without a huge performance penalty. Currently I'm hitting issues with the transactions not starting because they're waiting for the scheduler to move on, but Go sits there waiting on the result (even though it was started in a Promise 🤷). I did take a stab at local / session storage too, so that could be an option as well.

I appreciate the links! I'll have to take a look at these projects, very interesting indeed.

I do not get frustrated. I create workarounds. Mission statement: Fix WontFix.

Fair 🙂 My most constrained resource is time, so any of these options could be great – just have to find the time to work on them!

I am interested in reading and writing to and from STDIN and STDOUT. I do not have a use for IndexedDB or NativeIO or other browser-backed private origin (sandboxed) filesystems (we can get and change the data anyway if we really want to https://stackoverflow.com/a/36098618).

Your use case sounds solid to me. Once we can figure out how to move the Go installation out of RAM or add some other form of caching, then you should be good to go. (If you do want to take a shot at solving these memory problems, just let me know.)

[guest271314]

but it looked experimental to me? If it looks like it will hit stable eventually

Still experimental. I am not sure if shipped by default now with a flag necessary. The tracking bug should have information on the current status.

Some issues are the necessity to read the entire file when reading and writing https://github.com/WICG/file-system-access/issues/157, https://bugs.chromium.org/p/chromium/issues/detail?id=1084880

Currently I'm hitting issues with the transactions not starting because they're waiting for the scheduler to move on

Have you experimented with sheduler.postTask() and queueMicrotask()? That API and function could potentially be of use to you, though am not sure what the exact issue is. Chrome can cache substantial amounts of data, including wasm code the browser appears to generate itself. I usually launch with at least

--aggressive-cache-discard --disable-back-forward-cache --disable-gpu-program-cache --disk-cache-size=1 --v8-cache-options=off

flags.

Once we can figure out how to move the Go installation out of RAM or add some other form of caching, then you should be good to go. (If you do want to take a shot at solving these memory problems, just let me know.)

That would be useful. Yes.

I have largely solved the issue that began this process by creating virtual devices that I can access via getUserMedia() https://github.com/guest271314/SpeechSynthesisRecorder/issues/17#issuecomment-749875748.

A TODO is to write a "circular" or "ring" buffer so that we can write to the same memory location once we reach the end of the designated memory instead of just allocating N amount up front https://github.com/guest271314/webtransport/blob/main/webTransportAudioWorkletWebAssemblyMemoryGrow.js#L13. That is, allocate, e.g., 3 seconds of data, when we read that last 128 bytes for AudioWorkletProcessor begin writing at 0 again instead of continuing to write at 3.1 seconds, in theory we should then be able to implement an infinite stream using a single WebAssembly.Memory instance https://bugs.chromium.org/p/chromium/issues/detail?id=1161825. Something like 8192 total memory we write to using a ReadableStream or WritableStream, when we are reading/writing 8084-8192 we signal to writer to begin again writing at 0, ad infinitum, so that we are not allocating more memory, rather overwriting a finite minimal amount of memory necessary to complete that rotation without gaps of silence, that is this https://github.com/WebWeWant/webwewant.fyi/discussions/161 realized for audio.

[JohnStarich]

Thank you for your encouragement 😄 Nice! Glad you got past your issue 👏

I've had an epiphany. (I swear the longer I stare at something the less it makes sense, but in this case it helped. 😝)

Classic rookie mistake, I ran WASM function calls on the main thread as part of my debugging steps. Unfortunately, IndexedDB requires transactions to complete on a follow-up run loop – so running WASM funcs that wait on the FS in the main thread blocks and crashes the program.

Turned out the IDB file system is working quite well, which is nice. I made some progress with memory reduction via the go build cache:

The first profile was the "before" and the second is "after", showing only adding an IDB FS for the go-build cache shaved off 250 MB at minimum. Next step will be to download & unzip the go.tar.gz onto one of these mounts too.

It definitely slowed down time-wise, so that'll need to be optimized at some point too.

[guest271314]

That is still a substantial amount of computing resources used. When running a live session could cause a crash. Native Messaging and WebTransport provide a means to run the native code and get the result in the browser without compiling to WASM, which resides in memory. Ig Go is already installed and functional on your system there is no need to compile a WASM file, request that file, and run the code in the browser. The browser can just field STDOUT from the native application or shell script. For example instead of attempting to compile mkvmerge to WASM, we install MKVToolNix using tar bar or package manager and are not retaining the entire code in RAM https://github.com/guest271314/native-messaging-mkvmerge.

[JohnStarich]

Agreed, it is substantial. It can be significantly reduced with this IDB FS, just need time to work out the details.

There’s some nuance for how WASM is handling shared data in memory, so much of the data is being duplicated as it’s being loaded and stored.

I hear you on native file access helping out, and that could be quite useful. My understanding is that it’s still experimental and only offered by Chromium though. I’m focusing on a cross-browser and cross-platform solution at the moment, to improve the overall experience. Once RAM and task completion times are more reasonable, then I can look at adding that feature too. (And, of course, PRs are welcome if you’re just itching to do it now.)

[guest271314]

Native Messaging is supported at Chromium and Firefox browsers. File System Access API is a relatively new and different API.

With Native MEssaging you can post messages to and from a native application, e.g., see https://github.com/guest271314/native-messaging-espeak-ng where espeak-ng, opus-tools etc. is downloaded and built at the OS, on the usser local filesystem, used at the browser via messaging. quic-transport protocol over WebTransport provides a means to proscess data using Streams API (ReadableStream, WriteableStream, et al.). I was not able to achieve the requirements of capturing entire system audio, specific application audio, using WebAssembly alone, aside from use of WebAssembly.Memory.grow(). You have managed to include Go in the host. What is the memory usage when using the already installed, or downloaded and installed Go with messaging or WebTransport?

[JohnStarich]

Aha, I totally missed the whole Native Messaging idea in there, my bad. So you're interested in making your native app available to a Go program in go-wasm on stdin/stdout via MessagePorts? (edit: and it could help running native go binaries too)

That sure sounds amazing.

My only question is, does it have to be in a browser extension to access those native message APIs? The docs seem to lean that way, but I'm not familiar enough with the API.

[guest271314]

I filed this Chrome feature request https://bugs.chromium.org/p/chromium/issues/detail?id=1115640 for the ability to write to a WebAssembly.Memory instance within the native application or shell script and access that data in the browser. The concept is a MessagePort that has the capabilities of a TransformStream and Transferable Stream with the connection to and from the browser and native application.

Native Messaging provides a means to execute arbitrary code and get output using text messages. I achieve the requirements of capturing system and specific audio output using Native Messaging first, which necessitated several steps, then tried the same approach using the deprecated QuicTransport and then WebTransport, which so far is the closest implemented API to what described in the feature request.

What I am attempting to convey is that WebTransport with quic-transport protocol provides a means to stream writing to STDIN and read STDOUT as a stream. That could eliminate the substantial RAM usage that was occurring, where that is an issue when running the OS on RAM.

I filed this bug because you embedded Go in WASM. At the time I was trying to embed espeak-ng and pavucontrol in WASM. I am still interested in embedding and controlling mpv https://github.com/mpv-player/mpv in WASM, to at least compare to the WebTransport approach, where no WASM and module loading are necessary.

[JohnStarich]

Sweet, I hope it goes through. I imagine security will be pretty important in nailing that idea down.

For QuicTransport / WebTransport, are you using essentially a local network connection between the native app and the web app? Sounds neat. That may help with this use case, possibly. I think using today's available APIs (for a standard Chrome user, for example) it sounds like this would require they run a native app, so they'd lose the "just open it in the browser and start working" aspect of go-wasm. I'll definitely keep my eyes on these APIs you've mentioned though, they certainly show promise.

Update on reducing memory usage: I've got the page using barely above 250 MB now, on my local copy. I've even reduced the memory usage during compilation to nominal levels. All "disk" access slowed waaaaay down, so I'm working on that piece now. Startup times of 5 minutes are less than ideal 😉

On the bright side, after the cache is populated, the startup time is near-instant. Also now Safari is able to run go-wasm without immediately crashing due to memory usage.

[guest271314]

For QuicTransport / WebTransport, are you using essentially a local network connection between the native app and the web app?

For testing, yes.

The server can be remote, https://w3c.github.io/webtransport/#dom-webtransport-webtransport-url-options-url.

I've got the page using barely above 250 MB now

That can still be prohibitive when running a live OS on RAM.

quic-transport provides a means to perform the task locally, using the native application or shell script.

For example, if you already have Go installed on your machine, there is no need to use WASM at all to run Go from the brower.

It should also be possible to create the server with Go, and other programming languages installed, e.g., Try It Online https://tio.run/#, if the application requires Web request, and not the installed code, again, without potentially creating a duplicate install of WASM code in RAM.

[JohnStarich]

@guest271314 Agreed on memory usage still being high. There will need to be further optimizations in the future, since my time is limited. I've also spoken with @fivedots, so I'll be giving nativeIO a shot next.

I've wrapped up the current changes to enable persistence and (slightly) lower memory usage 🎉 You may need to drop caches and hard reload, but the new version is live: https://go-wasm.johnstarich.com

For example, if you already have Go installed on your machine, there is no need to use WASM at all to run Go from the brower.

This is certainly interesting, I'll need to give it further thought. Ultimately, if it over-complicates the user experience then I may need to leave it out. I don't know many people that would be willing to configure experimental features for a browser-based IDE. But hey, I could be totally wrong! 🙂

[guest271314]

Check this out https://blog.stackblitz.com/posts/introducing-webcontainers/

Again, these environments are not running on remote servers. Instead, each environment is completely contained within your web browser. That’s right: the Node.js runtime itself is running natively, inside the browser, for the first time ever.

[guest271314]

@JohnStarich FWIW a proof of concept of what I was trying to achieve when I filed this issue https://github.com/guest271314/NativeTransferableStreams.

[JohnStarich]

@guest271314 That's awesome! Thanks for sharing! Looks like they've managed to get a networking stack working, but also quite a lot of press and funding around the project.

I'm curious how they're handling non-native modules, or if that's even supported at the moment. 🤔 Lots to think about.

[JohnStarich]

Congrats! I may take a look around, seems like a cool project too.