option to treat `.js` files as StratifiedJS sources

ghost commented 7 years ago

This would allow the users of languages that compile to Javascript to use StratifiedJS’s features more easily.

In my particular case, I’m trying to run JS‐compiled Ceylon as StratifiedJS, to be able to use StratfiedJS’s features and functions from Ceylon. Currently, everything works beautifully, but I have to manually change the compiled JS files’ extensions, and therefore have to serve them myself (I can’t use Ceylon modules directly from Ceylon’s repository, the Herd).

I propose to have a treatjs attribute on required.

It’d accept three values:

js
sjs-sjs
sjs-js

js would act as it currently does today.

sjs-sjs would treat .js files as if they were .sjs files.

sjs-js would treat .js files as if they were .sjs files, but still resolve required resources as if they were .js files (i.e. resolve them absolutey, as opposed to relatively).

Then, we could use a module from Herd:

<!doctype html>
<html>
    <head>
        <script src="[foo]/stratified.js"></script>
        <script type="text/sjs">
            require.treatjs = "sjs-js";
            require.hubs.unshift(["herd:", "https://modules.ceylon-lang.org/repo/1/"]);
            require("herd:my/mod/0/my.mod-0.js").run();
        </script>
    </head>
</html>

afri commented 7 years ago

There is actually a somewhat ugly way to do what you are trying to do.

See working example here: https://github.com/afri/serve-js-as-sjs

It consists of two parts:

Firstly, there is an undocumented property extensions on the require function. This attribute contains a compiler function for each known extension. What we need to do is override require.extensions['js'] to call require.extensions['sjs'] when it sees an sjs file from your 'herd:' hub.

This is where the second part comes in. To be able to identify files from your hub in the compiler function, we have to make sure that the module id does not get rewritten. If you have a hub like require.hubs.unshift(["herd:", "https://modules.ceylon-lang.org/repo/1/"]); it just rewrites your module url to a normal https url. Instead we need to add a hub with a custom loader that uses http.get (i.e. XHR) to load your file manually.

Let me know if this helps!

ghost commented 7 years ago

Okay, this really helped! Thanks.

The only problem I had is that Ceylon was expecting require to resolve URLs relative to the Ceylon “modules” directory, but StratifiedJS’s require was resolving them relative to each file.

The way I solved the problem was by using $$ceylon$require. It is a function that JS‐compiled Ceylon will prefer to use, if present, instead of the regular require. I used it to prepend "herd:" to every require call it does, and to append ".js" if it’s not present.

Then, to be able to load a local Ceylon module (space.zambonifofex.sjs.hello/0) that depends on resources from the Herd, I added another exception to the require.extensions rule.

It looks like this:

<!doctype html>
<html>
    <head>
        <script src="https://code.onilabs.com/sjs/latest/stratified.js"></script>
        <script type="text/sjs">
            {
                (o=>(require.extensions.js = function(src, descriptor)
                {
                    if(descriptor.id.startsWith("herd:") || descriptor.id.endsWith("/space.zambonifofex.sjs.hello-0.js"))
                    {
                        return(require.extensions.sjs(src, descriptor));
                    }
                    else
                    {
                        return(o(src, descriptor));
                    }
                }))(require.extensions.js);

                $$ceylon$require = function(c, require, args)
                {
                    var arg = "herd:" + args[0];
                    if(!arg.endsWith(".js"))
                    {
                        arg += ".js";
                    }
                    return(require(arg));
                }

                require.hubs.unshift(["herd:", "https://modules.ceylon-lang.org/repo/1/"]);

                require("space/zambonifofex/sjs/hello/0/space.zambonifofex.sjs.hello-0.js").run();
            }
        </script>
    </head>
    <body></body>
</html>

The only two problems that remain unsolved are Ceylon’s:

The Herd doesn’t support CORS requests
There are three problematic accesses in ceylon.language: one .super access and two .package accesses. The problem is that Javascript allows .keyword accesses, but StratifiedJS doesn’t.

Patching ceylon.language and serving it myself works perfectly. I’ll request the Ceylon guys to fix this minor issues, and hopefully in the future we can transparently use StratifiesJS’s APIs and features from Ceylon.

However, this only works because there is a $$ceylon$require. I still think there should be some way to use .js files as if they were SJS files, but still resolve required urls as if they were JS files.

afri commented 7 years ago

The only problem I had is that Ceylon was expecting require to resolve URLs relative to the Ceylon “modules” directory, but StratifiedJS’s require was resolving them relative to each file.

It is possible to inject a custom require function into your loaded modules too, but again it uses an undocumented sjs feature, so your way of using $$ceylon$require is definitely preferred. But I can outline it to you if you want.

There are three problematic accesses in ceylon.language: one .super access and two .package accesses. The problem is that Javascript allows .keyword accesses, but StratifiedJS doesn’t.

The last released version of SJS is pretty ancient :/ If you use the version from the master branch (which is also at https://code.onilabs.com/sjs/unstable/stratified.js), .keyword accesses will work. Note that despite the name ('unstable'), I'd definitely recommend it over the released version ('latest'). We use it in production all the time. At some point we'll get round to making a new release...

ghost commented 7 years ago

But I can outline it to you if you want.

No, you don’t need to. Just knowing it’s actually possible in case someone else needs it is enough.

Since you recommend it, I’ll definitely start using the “unstable” version.

Thanks for you help!

(by the way, I have opened an issue on ceylon/ceylon-herd, asking for CORS support: ceylon/ceylon-herd#279)

onilabs / stratifiedjs

option to treat `.js` files as StratifiedJS sources #19