Document difference from GopherJS

[FlorianUekermann]

First off, I'm very happy to see someone having another go at this. I'm very impressed by what I've seen.

That said, most people will ask themselves how this project differs from GopherJS. I've been using GopherJS for a while in production now and don't feel like it is missing anything fundamental. Could you clarify...

• ...why I may want to choose Joy over GopherJS, now or in the future?
• ...why you decided to start from scratch?
• ...if you see a shared future for Joy and GopherJS

The answers probably belong into the readme and FAQ, since this is probably every new visitors first question.

[matthewmueller]

Thanks for opening this issue @FlorianUekermann. Definitely need to document this better. Will answer each of them better but for now, I'll copying and pasting what I wrote before here:

---

GopherJS is an awesome project and I gave it an honest go before working on Joy.

It seemed like GopherJS approached the compiler wanting to compile existing Go code to Javascript code. Getting big Go programs working in the browser is a huge accomplishment, but it's not the approach I took with Joy.

My approach to Joy was asking myself: Could I create a more productive frontend development environment in Go than by just using Typescript, Webpack, React and the Javascript module ecosystem? I believe the answer to that will soon be yes.

This question led me to an entirely different architecture that just didn't seem feasible to do in GopherJS without basically rewriting it.

Now, it's hard to say: If I had spent all this time just refactoring GopherJS, could it be the same project? The answer is maybe – I'm not really sure.

[FlorianUekermann]

It seemed like GopherJS approached the compiler wanting to compile existing Go code to Javascript code.

Does that mean Joy will not try very hard to be compatible with existing go code (maybe relaxed stdlib compatibility)? Or is compiling existing code one of multiple goals.

My approach to Joy was asking myself: Could I create a more productive frontend development environment in Go ... This question led me to an entirely different architecture

Could you elaborate a bit on that. Maybe an example of an architectural difference?

I saw that you put a lot of effort into efficient dom integration, which has a couple of rough edges in GopherJS. I assume this has some implications for how go interacts with JS variables. Is that something you found difficult to realize in GopherJS?

[matthewmueller]

Another important difference today is that GopherJS implements much more of Go's standard library and the syntax translation is more complete.

This will get filled in as we reach 1.0 and 2.0, but most existing Go programs that use the standard library won't yet compile. I'm hoping to get your help to make this happen faster!

Some links:

• More info about Go interoperability: https://mat.tm/joy/#go-interop
• Find out what's parts of Go don't yet compile in Joy: https://github.com/matthewmueller/joy/milestone/1
• To see what's in the stdlib today: https://github.com/matthewmueller/joy/tree/master/stdlib

[PaluMacil]

My takeaway from what you're saying is: "GopherJS started with Go as a priority and is working towards great JavaScript interoperability whereas Joy started with JavaScript as a priority and is working towards great Go interoperability."

Is that roughly correct?

Regardless, this type of project seems like a lot of fun regardless of whether it is used alongside or instead of GopherJS.

[matthewmueller]

@PaluMacil exactly right! That's a really nice way of putting it.

[matthewmueller]

Added a section on the website: https://mat.tm/joy#faq-gopherjs

Thanks @PaluMacil for that wonderful summary.

[myitcv]

@matthewmueller - just taking a look through the website etc for Joy - a very impressive release.

I've contributed a bit to the GopherJS project so am particularly interested in the comparison between the two.

The line in the summary above that caught my eye way:

"GopherJS started with Go as a priority and is working towards great JavaScript interoperability"

To my mind the JavaScript interoperability of GopherJS is very well defined and implemented (notwithstanding this open issue where there are certain bits that need tidying up). What particularly did you find lacking?

[matthewmueller]

@myitcv Hey dude! I saw you on quite a few issues for GopherJS, so I'm happy you've stopped by.

Let me try and get more specific and maybe you have some thoughts and solutions to these issues.

This is more tailored to what I found lacking about Gopher, but there's a bunch of stuff that's missing from Joy that Gopher handles beautifully.

1) Dead code elimination is a must if you want to have big and capable standard libraries like what Go has. The builds are just too big for basic things like fmt which everyone uses. That's maybe an infamous example with alternatives, but in general I wasn't able to find an easy way to generate small builds in Gopher.

2) A good DOM library is also very important for frontend developers, I tried using this one: https://github.com/dominikh/go-js-dom but found that I'm basically shipping another DOM: I already have one that's native to the browser and now I'm shipping one that's just wrapping all the native calls. Joy uses a macro system so window.AddEventListener(event, handler) just becomes window.addEventListener(event, handler).

3) For better or for worse, you really need to support React and it's variants. It's just such a fundamental frontend technology. I tried a couple reacts for Gopher. Actually just realizing... I tried your React! This might have just been me, but I think that stuff should just be baked into the compiler, rather than generating it. My north star for this is next.js, I think they did such a good job hiding the complexity of the JS platform and only exposing the good parts of JS. I'd like to do that with Joy and avoid complexity of something like codegen at the component level.

4) I personally think the channel implementation went way too far, though I may be totally missing something here. I think channels and CSP can be done with very little JS when you use something like generators or async/await, which can be compiled down to ES3 using regenerator.

---

My gut feeling for how this plays out is that Gopher will be a lot more compatible with existing Go code. Things like great pointer support, a more proper goroutine model etc. Joy probably won't go this way and will probably opt for compiler errors when it comes across those types of things (e.g. a map[struct{}]struct{}). So maybe Joy will be considered a subset of the Go language.

My aim for Joy is to be a great alternative to Typescript/Flow and my hope is that I can bring a bunch of JS devs to Go as they look for ways to build more stable web applications.

[shaban]

I can only talk about my reasons for being interested in joy. Maybe it's interesting to you guys.

1. working with gulp/grunt/npm is annoying, very slow turnaround times and compared to go infrastructure it's just a mess.
2. Since i am interested in using go as some sort of supervisor software to run all kinds of web related processes like sass compilation, frontend compilation, database operation, forms generation, i want to be able to compile javascript on the fly while making a request during debugging. I do that by using very shallow js-minification and file concatenation which means i can only use very common javascript since i don't have code morphing by babel etc. and i don't have .map support for easier debugging.
3. I want to be able to create something like web components. It need not follow the web components spec, but i basically want to encapsulate template and javascript in a way that gives me good editor support (basically html mode and js/go-mode). Looking at the virtual dom features of joy it looks like this should be doable.
4. From the documentation interoperability with javascript looks more convenient than in gopherjs.

So in a nutshell if i was to program a huge web-app i would probably go the go/webassembly way. But for almost everything else i would really love to have a sane frontend stack that does NOT depend on node and for the latter gopherjs is overkill and lacks a bit in the comfy department when it comes to js interoperability.

[FlorianUekermann]

I personally think the channel implementation went way too far, though I may be totally missing something here. I think channels and CSP can be done with very little JS when you use something like generators or async/await, which can be compiled down to ES3 using regenerator.

The reason is simple. You can't use or emulate async/await in event listeners and other non-async js calls. That means you won't be able to use channels or locks there without some kind of scheduler. That is particularly useful inside event listeners. The scheduler in gopherJS isn't even that complex, it just keeps lists of which go routine is waiting on which channel and executes it as soon as you do a send. So unless you produce some kind of deadlock you can use locks and channels everywhere. The feature isn't documented very well, but it works. There is some discussion here gopherjs/gopherjs#720.

Sooner or later you'll need this. Fortunately it's not that complex, now that gopherjs spelled out how to do it.

A good DOM library is also very important for frontend developers, I tried using this one: https://github.com/dominikh/go-js-dom but found that I'm basically shipping another DOM... Joy uses a macro system so window.AddEventListener(event, handler) just becomes window.addEventListener(event, handler).

This. So much. It's both a size issue as well as a performance problem.

Dead code elimination is a must if you want to have big and capable standard libraries like what Go has.

+1

[matthewmueller]

@FlorianUekermann thanks for the details and the link, the example you provided in that issue is a good test to try out.

You can't use or emulate async/await in event listeners and other non-async js calls.

Hmm, sorry I'm not quite following this. Why doesn't this work?

var btn = document.getElementByID("btn")
btn.addEventListener('click', async function() {
  document.write('oh hai')
})

http://jsbin.com/hurujularo/edit?js,output

Alternatively, maybe this?

var btn = document.getElementByID("btn")
btn.addEventListener('click', function() {
  (async () => {
    document.write('oh hai')
  })()
})

[FlorianUekermann]

You didn't put await in there. Using async functions is always possible (but rarely sensible in an event listener, since your code won't be executed in order anymore). If you just put await before the function call in the second example, it'll fail with Uncaught ReferenceError: await is not defined

Same is true for using await in any other synchronous context. So you can't use async & await to implement channels, if you want to be able to call go code from JS (including event listeners). I don't see a way around writing a simple scheduler until these design issues are fixed in JS (seems unlikely).

[matthewmueller]

Okay bear with me as I try to understand this problem better. I've read through the Gopher issue and also some of the outbound links. It sounds like what you're saying is that this won't always work:

window.addEventListener("keydown", async (e) => {
  var ch = e.key
  await sleep(500)
  var pre = document.querySelector("pre")
  pre.textContent = pre.textContent+ch
})

function sleep(ms) {
  return new Promise((res, rej) => {
    setTimeout(res, ms)
  })
}

http://jsbin.com/qehatuwume/edit?html,js,output

There's a chance the promises will get executed out of order resulting in your text no longer matching your input. And this is an issue because in this particular case, the order of callbacks matter.

Is that the problem you're talking about?

[FlorianUekermann]

There's a chance the promises will get executed out of order resulting in your text no longer matching your input.

If you use a random sleep time or a fetch instead of the sleep you'll probably see the characters out of order if you type fast enough. Not entirely sure in your particular example, since I'm not an expert on scheduling in JS.

But I don't think discussing the finer details of how the JS scheduler works is necessary for the discussion of a Go implementation. The problem is simple: If you use a normal event listener or JS call without async, you won't be able to use await, so you can't implement a channel receive by using await, nor any of the other synchronization primitives Go provides.

If you are wondering why someone may want to use non-async event listeners in the first place, consider that JS has a well defined event flow (https://www.w3.org/TR/DOM-Level-3-Events/#event-flow), which gives you a lot of useful guarantees to work with (serial execution of event listeners, ordering of event listeners, bubbling, cancellation, etc.). All of that goes out the window once you start using async. In practice you have three choices:

1. No channels in event listeners
2. Throw out the benefits of JS event flow and use async event listeners
3. Implement channels via a scheduler like GopherJS, not via await.

I personally like option 3 the best, but you may have different goals than I do.

[matthewmueller]

If you are wondering why someone may want to use non-async event listeners in the first place, consider that JS has a well defined event flow (https://www.w3.org/TR/DOM-Level-3-Events/#event-flow), which gives you a lot of useful guarantees to work with (serial execution of event listeners, ordering of event listeners, bubbling, cancellation, etc.). All of that goes out the window once you are start using async.

Got it – thanks for clarifying and I'll make sure to test for this.

The implementation is not set in stone by any means and it sounds like I'm mistaken about what the GopherJS scheduler is actually doing. We may very well end up going the GopherJS route here. You probably saved me some time and headache so cheers for that 🍻

[RangerMauve]

What about wrapping over DOM events with something like an event delegation setup.

• You provide Go APIs to listen on an event
• Wrap calls to addEventListener so instead of adding actual callbacks, you have an EventEmitter wrapping over top of it which you attach the callback to instead, and it only attaches a single DOM callback
• Your custom event emitter will await for a callback to execute, before moving on to the next callback
• This will ensure all event listeners will be executing serially relative to each other.

It'll add a bit more runtime, but it will support most use-cases without doing the fancy stack tracking that GopherJS has.

Since Go already uses goroutines in a lot of places, it's already equipped to handle coordinating async actions, so I don't think it'll be all that hard for somebody to listen on different events and coordinate between them with channels. Your code will look more like regular async-heavy Go in the end

[RangerMauve]

Something like domdelegate but with support for async functions like promise-events

[FlorianUekermann]

What about wrapping over DOM events with something like an event delegation setup.

Possible and not even very hard.

Whether you want to reinvent that part of the platform is a different question. Note that you will still have to trade in certain properties. For example, browsers freeze the interface until event listeners are done executing. To give an example why that is interesting: You can use that to guarantee that a click on a delete button will actually remove related UI before the user can interact with them. This isn't hard to work around, but it shows that there are trade-offs here. To me it is unclear which approach is the best. But it is quite clear that the usual JS&DOM semantics are what most people are familiar with.

[RangerMauve]

What about preventing the use of channels inside eventListeners by default (like you proposed), but providing this async API in case you do want to use channels inside eventListeners. Channels are used for coordinating goroutines which are inherently asynchronous from each other, so I think it makes sense to restrict async primitives to only be usable within async contexts. Using a channel outside of coordinating goroutines in regular Go doesn't make sense since it'd block your main thread forever.