golang / go

The Go programming language
https://go.dev
BSD 3-Clause "New" or "Revised" License
124.42k stars 17.71k forks source link

syscall/js: Make syscall/js optional when compiling wasm? #27766

Open justinclift opened 6 years ago

justinclift commented 6 years ago

At present, when we compile to wasm syscall/js is automatically included.

This forces the runtime to either be a browser (eg Firefox, Chrome, etc), or at least pretend to be one.

That's kind of non-optimal, as some of the wasm execution environments presently being developed aren't targeted at browser environments. eg:

In the WebAssembly Gophers Slack channel we have people asking about non-browser use cases fairly often. It seems likely that'll be a fairly standard use case, if it can be catered to as well.

How feasible would it be to have some way to suppress the default syscall/js inclusion, or otherwise make it optional?

Xe commented 6 years ago

I have investigated creating a custom GOOS in the past, but it fails with that attached gnarly linker error. Guidance on it would be helpful, as implementing this would unblock the use of Go for my server side webassembly things.

sbinet commented 6 years ago

Loosely related to https://github.com/golang/go/issues/25612 and github.com/go-interpreter/wagon

peterbourgon commented 6 years ago

Little to add except that I also have use cases for Go-to-wasm that don't involve a browser, and I'd be happy to chat about them to anyone interested in the issue.

bcmills commented 6 years ago

CC @neelance @cherrymui

neelance commented 6 years ago

If you don't have syscall/js, then you'd need something else to interact with the environment. Since there currently is no standard API in the WebAssembly ecosystem, I don't see why we should invent yet another API on our own. If such a standard API emerges in the future, I'd be happy to adopt it.

Today, I would implement the backend API of syscall/js as much as needed for the standard libraries, but no further. You don't need to implement full JavaScript for it. Do you see any significant downsides to this approach?

justinclift commented 6 years ago

@Xe @peterbourgon Probably best for you to jump in here. :smile:

Xe commented 6 years ago

@neelance for now I have been using https://github.com/CommonWA/cwa-spec as an imaginary system ABI. What calls would we need to add to this for Go? The time call is one of them.

neelance commented 6 years ago

Is there any effort to get https://github.com/CommonWA/cwa-spec adopted by https://github.com/WebAssembly, maybe as an official spec?

justinclift commented 6 years ago

CC @losfair

peterbourgon commented 6 years ago

@neelance

If you don't have syscall/js, then you'd need something else to interact with the environment.

Essentially I want to punt on this question. I want to compile Go down to a WebAssembly module, exporting a set of functions as a sort of public API, and binding to or declaring (somehow) a set of import functions, to consume other modules. I want to load and run this module in a server-side environment, with no relation to JS. CommonWA is not of much interest to me (us) in this context.

neelance commented 6 years ago

@peterbourgon

Would you want to call the functions concurrently or only one at a time? Which types of parameters and return values do you expect? Would they do HTTP requests to other services?

peterbourgon commented 6 years ago

@neelance I wouldn't take anything off the table in terms of functionality. If you're looking for some kind of examples, I can try to dig something up; let me know.

edit: To be clear, I would want to be able to answer each of those questions differently, in different modules.

Xe commented 6 years ago

@peterbourgon in this case using CommonWA (note: I am not married to CommonWA, it's just not something made by me and looks like it could easily be the basis we could build off of as a community to make it better for everyone) would us all just be standardizing on the set of things the binaries the Go compiler emits would be depending on. These things would then be implemented by the JS side of things too.

justinclift commented 6 years ago

@peterbourgon

If you're looking for some kind of examples, I can try to dig something up; let me know.

Please do, they'd likely help. :smile:

DavidHuie commented 6 years ago

I have a similar use case to @peterbourgon in mind. Wasm on the server creates the opportunity for new types of features, such as user developed, server executed extensions to products. I'd love to use Go for these sorts of things!

neelance commented 6 years ago

Sorry, but I still don't see why syscall/js prevents people from using non-JavaScript hosts. It is only an interface, it should be possible to implement it without having full-blown JavaScript.

Xe commented 6 years ago

What can non-JavaScript hosts do to stub it out or "break" all of its functionality?

neelance commented 6 years ago

@Xe I don't get your question.

Xe commented 6 years ago

@neelance What is the minimal implementation required for every function to have to return an error-like response?

neelance commented 6 years ago

@Xe That of course depends on the function. I would just start with an empty stub of each function which logs its execution and then add the implementation step by step.

Xe commented 6 years ago

Yes, can you please provide these empty stubs? I can't figure them out on my own apparently.

neelance commented 6 years ago

I don't even know which WebAssembly host you are trying to use.

neelance commented 6 years ago

The import functions that you need to stub are the ones you find in https://github.com/golang/go/blob/7a6ccece877232fc58872c85fdf0718629b83d91/misc/wasm/wasm_exec.js#L199

Xe commented 6 years ago

For all of these functions: https://github.com/golang/go/blob/master/src/syscall/js/js_js.s

These are things that I must know to implement these calls and have not been able to reverse-engineer on my own.

neelance commented 6 years ago

It's the normal Go calling convention. All arguments and return values live on the stack. You can find the function signatures in https://github.com/golang/go/blob/master/src/html/template/js.go and in the wasm_exec.js I linked above. You can also find the stack offsets in the wasm_exec.js if you don't want to calculate them on your own. A quick Google search got me this: https://science.raphael.poss.name/go-calling-convention-x86-64.html

neelance commented 6 years ago

C has a different calling convention than Go, so you won't be able to use simple C function signatures.

Xe commented 6 years ago

I realize the calling convention is different, I would translate that by hand. I want to just know what i should return for the values such that it will get error-like responses, consider go.runtimeNanotime:

The stack layout in relative ordinal bytes is something like this:

sp+0 -> stack pointer
sp+8 -> expected int64 return value

There are no arguments to this function and there is one int64 return that signifies an integer number. The zero return in this function is the value zero, as such:

func runtime.nanotime() int64 {
  return 0
}

The idea is that this kind of basic documentation about what the ABI expects (and what values are zero-like) will provide people who want to use Go in WebAssembly on the server side guidance on how to stub out functionality that they don't want to support (for example, the very reasonable case of being not a browser and not having a javascript VM involved, so Javascript interaction is not needed and thus removable). Can you please help?

neelance commented 6 years ago

@Xe Sorry, I don't have that right now. The os package will also require syscall/js to not return zero all the time. I can only recommend that you start with a simple example, add what's necessary to make it work and then continue from there. I can help you further if you have a specific error that you don't know how to resolve.

Xe commented 6 years ago

Then if it shouldn't return 0 all the time, what is the valid thing to do?

neelance commented 6 years ago

Fulfill the interface good enough that it supports the actions that you want to support or that it at least does not crash on initialization. Specifics depend on the case.

neelance commented 6 years ago

I can help you further if you have a specific error that you don't know how to resolve.

This still holds. I don't see how I can give you more information without implementing it myself. There is no specification that you can fulfill in one go. You'll have to start with some concrete example that you want to make pass and then we can figure out together what is necessary to do so.

neelance commented 5 years ago

I've written a proof of concept since so many people were asking for it: https://github.com/neelance/go_js_wasm_exec

justinclift commented 5 years ago

Thanks heaps @neelance. :smile:

backkem commented 5 years ago

Exploring options for supporting different types of IO in neelance/go_js_wasm_exec#1.

backkem commented 5 years ago

There is initial work being done on a CloudABI target at NuxiNL/go. This is the system ABI the CraneStation/wasmtime runtime plans to use.

justinclift commented 5 years ago

This is probably relevant as well: https://hacks.mozilla.org/2019/03/standardizing-wasi-a-webassembly-system-interface/

neelance commented 5 years ago

Awesome! This is exactly what I was waiting for. I'll look into it soon.

tiborvass commented 5 years ago

Does this mean we'll have a GOOS=wasi ?

peterbourgon commented 5 years ago

@neelance It was suggested to me that anyone working on wasi for Go might want to join #wasi on the Mozilla IRC—just passing that info along.

justinclift commented 5 years ago

31105 sounds related too. :smile:

mfateev commented 4 years ago

I believe for execution Go doesn't need any environment support. Even random seed and time can be hardcoded and it is fine for a lot of use cases. Forcing every host to provide a dummy implementation of WASI or syscall/js sounds very artificial. BTW when Rust and C compile to wasi they don't have any required imports.

Ideally I would love to have the compiler options to not include some Go standard library components at all. For example I would prefer that in some cases the code that references any file or socket IO wouldn't even compile.

neelance commented 4 years ago

I believe for execution Go doesn't need any environment support.

If there is no environment, then your Go code can calculate something, but then only discard the result.

when Rust and C compile to wasi they don't have any required imports.

Neither Rust nor C has a runtime/goroutines. You can directly call a Rust/C function from your WebAssembly host. This is not easily possible with Go's semantics.

mastersingh24 commented 3 years ago

TinyGo does not have this issue as they do not assume a JS host environment.

rsc commented 3 years ago

This issue is titled "make syscall/js optional" while #31105 is instead to assume wasi. As noted above, you have to have something or else there's no way to interact with the outside system. Perhaps this issue should be closed as a duplicate of #31105?

For wasi, https://github.com/golang/go/issues/31105#issuecomment-884696837 describes a way to use Go with it.

justinclift commented 3 years ago

Perhaps this issue should be closed as a duplicate of #31105?

Not sure personally. I don't use Go for wasm any longer, so no longer have any err... skin in this issue.

If other people do, I'll close or leave this open as seems appropriate. :smile: