Question: roadmap and some coordination questions

[mstoykov]

Hi, @dop251 (and whoever else is interested in the development of goja).

tl;dr: I (on behalf of the k6 team) would like to coordinate on getting goja to support more ECMAScript standards :D.

I am one of the developers of k6 which uses goja a lot ;). Previous to you starting the es6 branch I had very small hopes that at some point we will be able to get more native support of es6+ javascript and instead was up to update the babel+corejs combo in k6. This though has some problems for us - mostly corejs(2.5.1 in this case) adds up to ~2.3mb of additional memory which given that we run multiple goja.VMs adds up ... pretty quickly ... also takes some amount of time to actually run at the beginning. And core-js 3+ isn't any smaller so ...

Looking at the current support in the es6 branch and what is supported in core-js 2.5.1 shim it looks that after some testing we are probably good to drop it. As for the es7 stuff - they look not very useful for k6's use case and probably things we can live without, but if possible I am for adding them to goja.

Another thing that k6 specifically is interested in is let, const, and the import/export syntax support. This is because all our examples and the VAST majority of user scripts definitely do use those and without those babel gets into play and starts transpiling code ... which for some cases takes upwards of a minute or two, while goja compiles similar code in ms. To a lesser extent we are also interested in lambdas, template literals and classes as those are also fairly used across the board.

Additionally(although it is possible that this will get the highest internal priority) we are interested in Promises + async/await syntax as it will be beneficial for writing load testing scripts. And we have a bunch of issues that are blocked on the lack of event loop, and good syntax for it is probably going to be a big issue as well.

I haven't looked at the specification all that much (especially recently as we were pushing the infamous PR 1007 and 0.27.0, which is now out :tada: :champagne: ), so I don't know how much any of the above is actually going to take as development time and whether there are prerequisites that need to be met in order for some of those features to be viable. Also ... while I have delved in the goja code, I am definitely don't understand most of it.

So my question is do you have some specific roadmap you are going of ... like "I am going to do feature1 then feature2 and feature3 can be done in independently.", "I won't at all try with some obscure feature nobody cares about". So that we can coordinate on trying to not work on the same things :)

Also, are you up for at least merging PRs which add functionality beyond ES6, either syntax or just additional utility functions?

I have to add a disclaimer, that while we would really like all of those and are willing to work on them, it is unlikely we will be able to do all of them at least not ... soon :tm: .

p.s. Thanks for all the work on this awesome JS VM :tada:

[dop251]

My main focus at the moment is brining the syntax up-to-date.

Current roadmap:

• Merge es6 branch into master (after some additional checks).
• Implement block-scoped variables (i.e. let and const). It is a big challenge as it requires a lot of changes in both compiler and runtime.
• Do other things that should be a bit easier, like arrow functions, template strings, extended object literals, spreads, etc.

I currently have no plans of supporting classes, generators and async/await. At least not before everything mentioned above is done.

Unfortunately I can't give any estimates as it's not my full-time job.

As for contributing, check the README.

[nwidger]

My contributions have mainly been in the goja_nodejs repository, but I'm definitely interested in goja development as well! I would also be interested in support for Promises + async/await syntax. For my project, I am currently writing async/await code in TypeScript and using the eventloop package in combination with a Promise polyfill to be able to run the compiled JavaScript in goja. This works, and the recent changes in the eventloop package make it possible to more easily write Promise-based native modules in Go, but obviously it would be great if the polyfill wasn't necessary. Although, I'm not sure how native support for Promises in goja would work given that the eventloop is a separate package, but perhaps I'm missing something there. I'm also interested in import/export support and module support in general, I have a PR open to add the beginnings of the Node.js module resolution algorithm to the require package. From poking around the k6 codebase, it looks similar to the module loader in k6 but without the ability to load from URLs. I don't know if we could join forces?

[cemremengu]

I am interested in the advacement of this library as well however due to his workload/other priorities the maintainer does not seem to be very active/responsive or aproachable at the time being for a more rapid iteration cycle which makes me hesitant to contribute here. I felt that his answer to @MStoykov was a bit "cold" and not very open to cooperation (no offense).

I wonder if forking or sponsoring can be a solution at this point to get up to speed.

[dop251]

Neither Promise nor async/await require an event loop. In fact, Promise should be relatively straightforward to implement, and I might do it shortly. async/await require a functionality similar to what generators require, which is far more involved.

[mstoykov]

@cemremengu I don't think @dop251 was cold or that they are unopen to contribution. As far as I can see this is the usual open-source project problem - contributions should be of reasonably high quality in order to get merged. Even as this is literally my job I also prefer if people contribute code, with tests and with a clear idea of what it needs to do, and preferably after they've communicated that they want to add 2k lines of code to the project, adding something that the maintainer(s) of the project doesn't agree should be there.

Also ... forking the project before we even try to make PRs, seems like an extremely bad idea. (no offense)

Apart from otto, there appears to be a single other JS VM implementation in golang - gojis, which looks dead before it even became viable, at least for what k6 needs. All of this tells me that making and maintaining a JS VM might not be ... all that easy :D.

So far @dop251 has been pretty fast in fixing bugs that k6 has had or merging (small) fixes that I've contributed, so I don't feel in any way that I am not welcome. If anything, for someone who isn't paid to do this, and such a complex project, I am impressed with the amount of time that is being put into it.

Of course, I would prefer if magically we get ES2020 in two months, but this is unreasonable unless someone (who knows what they are doing) is getting paid to work on it fulltime ... and even then it will probably be a stretch :D. So I hope there are more people who start contributing, especially now that there has been some movement towards ES6. If such a person comes around - I guess we can talk about paying them somehow ;)

On a more related to the topic note:

@nwidger As far as I understand "import/export" does not have anything to do with nodejs or even ... loading files ... it is for the most part (if I remember correctly from my reading of the spec a year ago) literally a semantic that says that loading resource with a specifier should return a structure, that can then be used to reference parts of another module. And that loading the same specifier or the resource with that specifier should happen only once. What the specifier means, or how it's loaded is not part of the spec.

I (personally) don't want nodejs resolution in k6 for example :D (and I don't think it should be part of goja, or at least it should be configurable). The reasoning behind this is that ... well, I think (as the nodejs creator and other people as well) that the nodejs module resolution algorithm ... is ... a bad idea :D. So bad ... they don't actually resolve ECMA Script Modules with it ;) I actually have no idea ... what the good qualities of it are, so I have really hard to time to express exactly what I think is wrong with it ... because I just think it shouldn't have been done :D.

I will try to write an issue with a proposal for how I think it should work inside goja, when I read the spec again and will try to get it in goja - probably ... in the next two months :sob:

[nwidger]

@MStoykov Ah okay, I guess I was mistaken and assumed that import works similarly in ES6 as it does in TypeScript. All I know is that the TypeScript compiler by default definitely uses the NodeJS resolution algorithm for resolving modules referenced via import statements. My PR in the goja_nodejs repo is just trying to make its require be able to find the same modules as tsc does. I'm relatively new to both TypeScript and (non-web) JavaScript development, so sorry if I'm muddying the waters here!

[luisfurquim]

can goja run webassembly?

[jonny7]

Hi @nwidger

Not to derail this thread, your project of TS with Async/Await mentioned above, seems what I'm trying to achieve, would you have any open-source repos to check out?

[nwidger]

@jonny7 Unfortunately all the work I did for that was for work, none of which is open-source. At one point I did see a project on GitLab that was called "goja-promise" or something like that that I was going to link here, but I'm having trouble finding it now (this was many months ago).

[jonny7]

Thanks @nwidger I appreciate the reply. Will see if I can find

[na--]

FWIW, if goja has some support for classes (ES6 only, not the latter additions) and basic import / export support (e.g. just parsing the syntax and calling some simple callback like ResolveModule(string) goja.Value or something like that, which goja users have to provide an implementation for), it seems like esbuild can probably be used to "support" a lot of the newer ES features: https://esbuild.github.io/content-types/#javascript

This would probably be a much simpler and better-performing thing to implement than messing with Babel. Considering esbuild is written in Go, it seems easy enough to integrate in the process: https://pkg.go.dev/github.com/evanw/esbuild@v0.13.6/pkg/api#Transform

(prompted by this thread that popped up on HN today: https://news.ycombinator.com/item?id=28860713)

[dop251]

What about promises and generators? I believe these plus classes are the only major ones remaining for ES6, will focus on them next.

[na--]

:+1: Awesome! :tada: Yeah, missed promises and generators, though I don't think esbuild will require them to support its other syntax transformations :man_shrugging: