Splitting of source files?

[pavel-krivanek]

Do you think that it would make sense to split the source files into smaller pieces for easier source code management and organization? I tried to extract the internal plugins, interpreter, primitives etc. into standalone files as you may see here:

https://github.com/pavel-krivanek/pavel-krivanek.github.io/tree/master/pharo8/js/squeakjs

I guess it would make the managing of the code harder with Lively-Kernel but it seems that it is unfortunately dead these days and the next Lively versions like lively4-core are very different. So the better support for standard JS tools should be considered.

[ErikOnBike]

Please also have a look at my suggestions for separating out some code to allow headless usage in NodeJS then. I'm running SqueakJS headless and did some refactoring. Have not heard back how to offer this as PR (except that it seems interesting). (See https://github.com/bertfreudenberg/SqueakJS/issues/91)

[codefrau]

Yes, I guess it's time to get rid of the Lively dependency. It's a shame though, live-editing a VM was a lot of fun (and it still works!)

Feel free to rearrange the source code and get rid of the Lively module system.

[ErikOnBike]

@bertfreudenberg

Two questions?

• do you want an active removal of the Lively module system?
• do you have time to accept and integrate PR's in the coming weeks? (*)

One question for all:

• how can we continu in an organised way? @pavel-krivanek and I (@ErikOnBike) made changes to the module structure and another PR from (@gkgoat) for integration with ObjectLand is pending which seems relevant. And I know @ccrraaiigg is also busy in this area.

I'd like to see things get integrated before we have too many divers implementations. Seems like a nice 2020 intention ;-).

(*) I'm asking since working on integration will not be worthwhile if it will not be accepted in your repo. I do not like this beautiful tool being forked all over, each going its own way. I have two pending PRs a couple of months old now, so I assume you are still busy with other things. Hope you do have time for continuing with SqueakJS. I have a feeling it still has a very bright future :-).

[codefrau]

Yes, restructure the code to use ES6 modules. Since this is standardized it should be relatively future-proof.

And yes, I will review PRs.

For discussion, we can use the vm-dev mailing list. There is also the Squeak Slack which has a #squeakjs channel. See https://squeak.org/community/

Here's to a great 2020! Happy New Year 😊

[ErikOnBike]

I'm looking into ES6 modules at the moment and find that browser support is limited in certain areas. I see for example that modules for WebWorkers are not supported (at least in my Chrome and Safari browsers). We can use something like webpack or rollup to create bundles which can be loaded in the browser (similar to the Google Closure Compiler which I see in use in the Makefile in the run folder). Any preferences? Leave it up to the 'user'?

[ErikOnBike]

I'm trying out rollup at the moment for a simple naive approach. Seems to work.

[codefrau]

I think the VM package itself would just use ES6 modules, and the demo app (html page) would use a bundler to build a version for the web, maybe? And that's what we would deploy as the gh-pages branch (https://squeak.js.org).

[ErikOnBike]

I'm testing some stuff at the moment. It looks promising. Seems okay to have regular ES6 modules and offer bundled versions (or let user create themselves) for browsers without (sufficient) support. So for compatibility on the gh-pages branch, we could indeed use a bundled version. I'm personally using a WebWorker version of a headless SqueakJS (*) (I think Craig was using that as well). This does require a bundled version (albeit with different modules) because of lacking broad support for ES6 modules in WebWorkers I think. (*) This means without using a number of display and audio plugins.

[ErikOnBike]

Is the VMMakerJS still valid/used? I also updated the code to work without the 'previous' module system (just updated the Smalltalk code in a text editor), but do not know on what image this should be tested. Also, some plugins seem to have had some manual changes (after first generation).

[codefrau]

VMMakerJS needs to be updated to work with the latest OpenSmalltalk VM sources. I only used it to translate all the plugins once and haven't looked at it since. That was 5 years ago - I thought I had written down the corresponding VMMaker package version but I can't find that :( Our best bet might be going by the timestamp in the VMMakerJS Monticello package and using a VMMaker package from around that time.

And yes, bugs like 1076d2c9981b9766a439a031e9f3e5ab318e4302 should have been fixed in the translator rather than the generated code, but at this point it's probably easier to hack the generated sources than to make VMMakerJS work again? Not sure.

We also still have VM bugs like #23 which I have not tracked down - might be in the VM code or in one of the plugins.

[ErikOnBike]

Short update: I think I have something working now. Both browser and NodeJS will run SqueakJS using the module system. I kept things simple (straight full module import, no explicit named imports). The different examples (scratch, Etoys, standalone) all seem to work well (on my machine ;-). I'll try to do a PR later this weekend. It is late over here, doing a PR now is probably not a good thing. Cheers

[ccrraaiigg]

Not having to use webpack and its ilk is one huge reason why I’m interested in SqueakJS in the first place. I won’t go there.

...Oops, darn, I guess I would go there if it could be done in a livecoding style from within some running SqueakJS system... e.g., some version of a mainstream bundler that can be loaded dynamically from a CDN and does everything that people expect from a bundler. I've even seen emscripten run this way.

[codefrau]

Chrome nowadays runs ES6 modules natively, no transpilation needed for developing the VM (that is, SqueakJS itself). For deployment (just using SqueakJS instead of hacking it) we transpile all modules into a single JS file that's compatible with more browsers. IMHO that's a good trade-off.

[ErikOnBike]

Sorry did not manage to do the PR this weekend. Still flattening a few wrinkles. I'm currently able to do a single CLI command (no config files) to create a simple bundle for the required files using rollup. Command: rollup squeak.js --file squeak_bundle.js --format iife. I have added some simple nodejs and headless (for browser) Javascript files for developers wanting more than starting an existing image in the browser. The headless script will also run in a WebWorker. I added a very small image (135kb) as demonstration.

[ccrraaiigg]

...just using SqueakJS instead of hacking it...

For myself and my clients, just-using-SqueakJS inevitably becomes hacking-it before too long, and we think that's a good thing. :) It's another one of those "preserve this vital option" things, like the Squeak VM simulator and code generator, the Lively-hosted SqueakJS VM devtools, and the all-in-one native Squeak app format.

[ccrraaiigg]

Erik writes:

I'm personally using a WebWorker version of a headless SqueakJS () (I think Craig was using that as well). This does require a bundled version (albeit with different modules) because of lacking broad support for ES6 modules in WebWorkers I think. () This means without using a number of display and audio plugins.

For what it's worth... I did my own initial WebWorker-hosted invocation of SqueakJS a while ago, from within another instance of SqueakJS, putting an "all-in-one" vm.js into a Blob and creating the worker from that. This was for creating a full live-object-system replacement for the change/sources files, part of the Naiad history system (https://thiscontext.com/naiad).

[codefrau]

Erik's code splitting was integrated, so we can close this now.