all: WebAssembly ("wasm") support

[bradfitz]

WebAssembly ("wasm") is similar to Native Client, but different notably in that other browsers plan to implement it.

http://webassembly.org/

This has been asked about a few times, so this is a tracking bug for it.

Whether we get it via cmd/compile, gccgo, or llvm-go, we can post updates here.

[minux]

@cherrymui and I discussed the possibility of a wasm GC port at length in the past few days:

Our current conclusion is that there is no efficient way to implement setjmp/longjmp functionality currently, therefore it's not a viable target of a gc port. We need to wait until it gets real stack unwinding and exception handling support.

All the other aspects looked fine, and we even designed a Go ABI under wasm (pass g and SP on wasm stack and everything else on emulated Go stack) and a way to modify the current MIPS backend to support wasm by emulating MIPS registers as wasm local variables.

Perhaps GopherJS can support wasm easier.

[ernado]

From gopherjs/gopherjs#432:

However, it is not a technology that GopherJS can use. GopherJS compiles on the AST level, not on the machine code level.

WebAssembly could be used as a backend for the normal Go compiler

[4ad]

@minux The lack of threads or async i/o seems to be a bigger problem than the ABI workarounds we'd have to do because of wasm quirks.

[sbinet]

@minux, @cherrymui could you post somewhere in more details what you did and achieved? as you may know the go-interpreter community is considering writing an interpreter based on either LLVM or wasm bytecode. (we may even have a GSoC student working on the wasm bytecode interpreter [1], so, just the "bytecode consumption" part, not the "bytecode production via some Go-based toolchain" one)

minimizing the amount of duplicated efforts and impedance mismatch between various components would be great.

[RaananHadar]

I highly recommend this as a strategic feature. Consider that in just a few years web programmers will flock in hordes to pick their language of choice to be compiled into web assembly. The sooner we join the race the better. Mozilla is pushing Rust hard as the web assembly language of choice...

[tiborvass]

I second what @RaananHadar said. It would be a pity if the spec didn't suit Go's needs, especially given how Go's runtime can be unique.

EDIT: Apologies for the MeToo nature of this comment :) Thanks for the pointer Brad.

[bradfitz]

Reminder: https://golang.org/wiki/NoMeToo

[vibhavp]

The WebAssembly stack machine context consists of linear memory and an operation stack. IIUC, saving the VM context can possibly be done by saving the:

1. Current PC, bytecode string
2. Linear memory, and the
3. Operation Stack

To a global list of saved contexts in the VM context struct, if any.

Thus, a setjump should simply restore these values, and continue with the interpreter loop as usual (This is similar to how the emacs bytecode VM implements signals: https://github.com/emacs-mirror/emacs/blob/master/src/bytecode.c#L785). An exception system can be implemented on the top of this, though I'm not sure how performant would this be.

Further, the webassembly spec mentions that the stack height at any moment in the bytecode is statically known, making all stack operations equivalent to operations on registers unique to each position on the stack. This paper describes a way to achieve such a stack -> register mapping for converting stack machine code to native code, allowing us to use setjmp/longjmp as usual.

Edit: What about using panic() and recover() with special WasmException{} values for signalling an exception? recover does the tough job of stack unwinding already, it shouldn't be hard to restore the interpreter loop after recovering from a caught exception.

[minux]

Yes, basically the discussion with @cherrymui resulted in similar findings.

The initial design with @cherrymui is this: (this stage focus on getting $GOROOT/test/sieve.go to work, no async IO is considered.) we reuse the MIPS port, mapping the 31 MIPS registers as WASM local variables and only use WASM stack for intermediate results. We can't use WASM's function call mechanism for most Go functions because the lack of stack unwinding support. change the mips objlink (cmd/internal/obj) to emulate each MIPS instruction with respective WASM instructions. This makes setjmp/longjmp (aka runtime.gogo) implementation trivial and reuses a lot of existing effort.

Go already needs to implement its own stack, separate of the WASM stack, because Go needs copyable stack, stack pointer maps and stack unwinding support, which are not available from WASM. (This strategy also aligns with the LLVM port, we can't use LLVM stack for almost the same reasons (unless we add a custom backend pass to LLVM proper to encode stack pointer maps).)

We will probably need to use a big switch to thread indirect jumps? This is similar to NestedVM's approach. http://nestedvm.ibex.org/

If we decide to implement a native WASM backend, then we can do this: the Go ABI is: all parameters (including results) on (emulated) Go stack, at function entry, the following are on the WASM stack: emulated SP, g.

The discussion happened more than one month ago, and I probably forgot some details at this point.

[vibhavp]

IIUC, you can completely discard the WASM stack (according to the spec). An extendable and copyable go stack is still something that needs to be investigated.

[minux]

Yes, the initial design I mentioned only uses the WASM stack for intermediates used in computation. All data is either on Go heap (linear memory) or pseudo registers in WASM local variables.

I don't expect WASM spec to incorporate moving/segmented and copying stack. It's rarely used by mainstream languages targeted by WASM.

[SerkanSipahi]

@bradfitz @minux @vibhavp What is the current state of wasm in golang? Nothing has happened since March 13th ! will it be possible transform golang to wasm in the future? is there maybe any route-map of that?

[kanocz]

if somebody will decide to implement wasm support I can offer my time for non-global tasks

[bradfitz]

@SerkanSipahi, nobody is working on this. This bug is marked LongTerm. If something starts happening, you'll see updates here.

[joeblew99]

A good roadmap ?

We have DELVE and GDLV for debugging golang with a GUI. It works well too on all Desktops and Perf is excellent. https://github.com/derekparker/delve https://github.com/aarzilli/gdlv

Now, GDLV is based on NUCULAR which is shiny with some nice abstractions. https://github.com/aarzilli/nucular

SO i was thinking this would be a good basis to do what has already been done here:

• Demo: http://shi-yan.github.io/AssortedWidgets/
• Code: https://github.com/shi-yan/AssortedWidgets

There are 5 WASM go libaries now too: https://github.com/search?l=Go&q=webassembly&ref=simplesearch&type=Repositories&utf8=%E2%9C%93

[techtonik]

NaCl is shut down - https://blog.chromium.org/2017/05/goodbye-pnacl-hello-webassembly.html So this should probably be cleaned up - https://github.com/golang/go/tree/master/misc/nacl

[ianlancetaylor]

I believe that NaCl is used by various parts of the Go ecosystem. That is, it's not used only by Chrome. And in any case that blog announcement is about PNaCl, which, although it provides functionality similar to NaCl, is actually a completely different product based on a different technology. So removing NaCl support from Go is premature at present.

Any in any case whether or not we remove NaCl from Go has nothing to do with whether we add support for Web Assembly.

[techtonik]

I would say that PNaCl is just an extension over NaCl to provide platform independent code https://developer.chrome.com/native-client/nacl-and-pnacl I asked about NaCl deprecation here - https://groups.google.com/d/topic/native-client-discuss/wgN2ketXybQ/discussion

If is possible to get list of those various parts of the Go ecosystem where NaCl is still used? Going through it one by one will make transitioning to WebAssembly more interesting.

[4ad]

I would say that PNaCl is just an extension over NaCl

You can say whatever you want, but it doesn't mean you are right. They are completely different ISAs, NaCL is just the target ISA (x86/arm) with some restrictions, while PNaCl is a completely different ISA for an abstract machine.

Go has never supported PNaCL, and the Go NaCL port was never suitable for Chrome anyway. It had nothing to do with Chrome, it was not usable in the browser. Whether Chrome abandons NaCL or PNaCL has no relevance to Go whatsoever. Nada, nil, no relevance. How many more times this must be said? Of course, if NaCL is abandoned completely then Go will be forced at some point to abandon it too.

WebAssembly support in Go has absolutely nothing to do with NaCL. Go might or might not get WebAssembly support. If, or when it might get such support has no relevance to the state of the NaCL port.

[bradfitz]

Here is a very important part of the ecosystem where we use nacl: https://play.golang.org/p/MfJIq8wb5-

(that runs server-side, not nacl-in-a-browser)

[techtonik]

Whether Chrome abandons NaCL or PNaCL has no relevance to Go whatsoever. Nada, nil, no relevance.

And who will support sandbox loader and toolchain then? It is imported from Chrome project, which switched efforts to WebAssembly. Wouldn't it be wise to follow and see if Go playground can already be ported from NaCl sandbox to WebAssembly sandbox? It can also be run server-side.

[ianlancetaylor]

I think everybody is in favor of supporting WebAssembly.

The time to discuss moving away from NaCl is after WebAssembly is fully working. There's no point to discussing it before then.

[vine77]

@ianlancetaylor, can you clarify "after WebAssembly is fully working"?

At a glance, it appears webassembly.org says "the initial version of WebAssembly has reached cross-browser consensus" and wasm is available in current or future browser versions from all browser vendors.

[ianlancetaylor]

@vine77 I mean after WebAssembly is fully working for Go programs, at least as well as NaCl works today.

[techtonik]

So what is the next test that should pass for WebAssembly to work for Go programs?

[ianlancetaylor]

@techtonik I feel like there is some confusion here. When we say that WebAssembly should work for Go programs, what we mean is that the Go compiler should generate WebAssembly code that can run in the browser. We mean that you should be able to write something like GOOS=wasm go build hello.go and get a program that can run inside a browser. We aren't even remotely close to that. We haven't even started. So there is no "next test." There is a lot of work to be done. And, as far as I know, nobody is actively working on it.

[joeblew99]

@ ianlancetaylor There are 4 implementations out there for golang. One seriously impressive. See my previous comment for the links.

Boy this thread is like Chevy Chase car seen with the kids in the back seat screaming " are we there yet " :)

[randall77]

@joeblew99 : none of those links are things that can compile Go to WebAssembly. They are things like a WebAssembly -> amd64 compiler written in Go.

[joeblew99]

I was afraid you would say that. Fair enough. So what IS needed for the " bring up" ? So everyone knows... Gopherjs kind of made the same mistake of not being at the right level in the compiler tool chain architecture / code generation ?

[randall77]

There's a long list. Most have to do with garbage collection, one way or another.

• Generate WebAssembly code in the SSA backend. The backend is used to register machines, adapting to WebAssembly's stack machine will take some work.
• What's a pointer? WebAssembly doesn't have a pointer type.
• How do we introspect the stack? We'd need this for garbage collection, certainly, but also panic/recover, traceback printing, etc.
• How do we suspend/resume a goroutine?
• How can we lay out the heap? WebAssembly provides only essentially sbrk. We need more general address space layout. Where do we store information about spans, ptr/nonptr bits, etc.?
• Multithreading. WebAssembly has no notion of threads at the moment, which we would need to, for example, do concurrent GC. We'd also need locks and maybe atomic operations.
• How do we provide access to the outside world for packages like os and net?
• WebAssembly doesn't have support for "goto".

That's probably not a complete list.

[techtonik]

When we say that WebAssembly should work for Go programs, what we mean is that the Go compiler should generate WebAssembly code that can run in the browser.

@ianlancetaylor would it be easier to target non-web first? NaCl was also designed to be run in browser first, but then got browser-independent loader, which Go currently uses. WebAseembly without browser - http://webassembly.org/docs/non-web/ - and it speaks about minimal shells for testing.

@randall77 is it possible to bring an actionable checklist out of this list? Like a master issue with links to issues that research each aspect in detail further?

[sbinet]

@techtonik wrt non-web: @vibhavp is working on that in the context of the GSoC internship I was alluding to a few posts above. it's over there: https://github.com/go-interpreter/wagon

[randall77]

@techtonik: I think the right place for such a checklist is a proposal document about supporting wasm. Once such a proposal is accepted and/or people are actively working on it, they could use issues for individual tasks, sure. That's premature at this point - I know of no one actively working on either (a proposal or code).

[joeblew99]

I agree about a proposal doc. All the I do / contradictions are in this thread and just needs it wrapped together, with a roadmap that lets stages the risk / reward balance.

I would also like to see it address graphics too as it's sticking out like a sore thumb for me.

[neelance]

I'm the author of GopherJS, a compiler from Go to JavaScript. I have written that project because I strongly believe that there should be alternatives to using JavaScript in the browser, alternatives like Go and others. I have decent knowledge on compilers, SSA, machine architecture etc., but I'm probably missing a lot of details, since I haven't written a compiler backend for machine code yet. This is why I don't feel qualified to write a full proposal doc. Still, I'd love to get some critical feedback on my thoughts about Go and WebAssembly.

While looking at WebAssembly, it strikes me as being very different to the usual machine code targets like x86 or arm. For example its architecture is a stack machine instead of a register machine. This means that it isn't immediately suitable as simply yet another target at the last stage of the Go compiler next to x86 and friends. One solution might be to not put it there, but instead have a significantly different approach on generating WebAssembly. The code for this approach would need to live besides the existing compiler stages, which would be not nice at all. One might even consider a fork of the compiler in this scenario. In short: I don't see this happening.

There might be an alternative: Emulate what we need. We may use the stack machine to emulate a register machine and hopefully do so in a reasonably performant way. WebAssembly has linear memory with load and store instructions, which is good. We would not use WebAssembly's call instruction at all and instead roll our own stack management and call mechanism. Stacks would live on that linear memory and be managed by the Go runtime. The stackpointer would be a global variable. All code would live in a single WebAssembly function. The toplevel would be a giant switch statement (or WebAssembly's br_table based equivalent) with one branch for each function. Each function would have another switch statement with one branch per SSA basic block. There are some details that I'm omitting here, but in the big picture this looks like a decent register machine to me. Of course the performance of this heavily depends on how well WebAssembly can transform these constructs into actual machine code.

So please tell me: Am I completely naive and nuts? Then I'm happy to postpone my thoughts until I've learned more. If not, then this may serve as a starting point for some actual proposal doc. Thanks.

[nathany]

In addition to all the work needed on the Go side, there are some things WebAssembly may need before the Go runtime can target it.

WebAssembly doesn't yet have threads, but it's on the roadmap and there is a spec. https://github.com/WebAssembly/threads

Regarding garbage collection, it sounds like there could be multiple options in the future, based on Lin Clark's talk https://youtu.be/HktWin_LPf4?t=28m31s

28:31 So, today, you can ship down your own garbage collector with code if you want to but is slow for a few reasons and the community group is making it possible for WebAssembly code to be used with just the built-in GC which is a highly optimised one that the browsers have been working on, so it will run fast and you will have that integration.

As with gomobile, there is the language bridge to figure out.

[neelance]

@randall77 also mentioned threads as a requirement for the GC. Is this really a hard requirement for the first version? Concurrency can also be done in a single thread.

Regarding the implementation of GC my personal opinion is that I don't believe in the one GC to rule them all. I'd rather prefer for Go to have a Go-specific GC and for Python to have a Python-specific GC. By completely managing our own heap and stacks on WebAssembly's linear memory we should be able to use the GC that Go already has, as far as I can currently see.

[randall77]

@neelance Yes multiple real WebAssembly threads are a nice-to-have, not a strict requirement.

[pmaddams]

The GC proposal for WebAssembly is still a work in progress. Interested parties can participate here:

https://github.com/WebAssembly/gc

[ghost]

@neelance, I know nothing about compilers, SSA, machine architecture, etc. Can you please tell how what you've described would affect binary size? If I'm understanding correctly the size efficiency was one of the high level goals of WebAssembly. It should be one of the priorities for this Go->WebAssembly compiler as well, right?

[neelance]

@alxkchr My experience is that the repetition due to code-gen boilerplate can be compensated a lot by applying gzip. Still, my proposal is not the most efficient solution with respect to binary size. Still, some solution that can be implemented in a reasonable amount of time is better than no solution, right? ;-) Also for me personally, binary size is not one of the first priorities.

[neelance]

fyi: I have started implementing this.

[SerkanSipahi]

@neelance best news ever :) do you follow the spec? (i hope)

[neelance]

The Go spec? It is a new backend/target for the normal Go compiler, so the spec is already in there. ;)

[SerkanSipahi]

i mean wasm spec :)

[stuart-warren]

https://github.com/neelance/go/tree/wasm-wip ?

[cznic]

@neelance

fyi: I have started implementing this.

Can you please share what approach will be used to address

Our current conclusion is that there is no efficient way to implement setjmp/longjmp functionality currently, therefore it's not a viable target of a gc port. We need to wait until it gets real stack unwinding and exception handling support.

(https://github.com/golang/go/issues/18892#issuecomment-276858667)

[sbinet]

@neelance: I know it's still early days, but I think it would be wiser to name the wasm GOARCH wasm32 (ditto for the packages)

[dmitshur]

@sbinet It's worth considering that the ARM architectures/packages are named arm and arm64 (similarly mips and mips64) rather than arm32 and arm64. However, I don't know whether that should be considered a good or bad example.

[neelance]

@SerkanSipahi I'm initially targeting V8 (nodejs/chrome) and using https://webassembly.github.io/spec/ as documentation on what to do.

@stuart-warren Yes, my WIP is on that branch. However, don't try to use it yet, it can't be easily built right now and it is full of copied code and stubs/hacks. I'll announce it here when it reaches some alpha state.

@cznic As written above, I am not using wasm's call instruction at all, I'm managing my own stack on the linear memory. Thus setjmp/longjmp can be implemented.

@sbinet Actually it will be a 64 bit architecture since wasm has 64 bit operations.