Module system

[joskuijpers]

In the recent light of #8, I should make an issue on the module system that I have in mind.

I want to implement the node.js-like module system:

var link = require("link");
link.concat("a","b");

A module looks like this (it is always a separate file, which increases quality of code and makes code style better):

exports.concat = function(a,b) {
   return a+""+b;
}

This exposes the concat function in the module. So you can choose public and private functions.

What are the advantages of such system with encapsulations?

• No more huge globals: the global scope in a module is the module scope.
• No predefined namespaces, they can be named by the user.
• No name collision (not that we have such problem with libraries, but it could happen when someone has a player object named Link and uses the Link library).
• Cleaner code.

Internally, require() (and include() for that matter) calls upon System.resolve("link","js","scripts") to find the actual path, and then calls into native to load the data. When loading the javascript, one can't simply do eval(). First off because eval is extremely evil. And next, because encapsulation is needed. Each module must also be passed the location of the module and its name. I will write more about this.

The actual implementation will probably depend on your java script engine.

[FlyingJester]

I don't really like the System.resolve idea. It seems like a lot of unnecessary boiler-plate to me.

[joskuijpers]

Code in engine, but it does resolve the library stuff. And it is only code in the engine, not in the game itself. It is also used for the file-system sandboxing.

[FlyingJester]

I don't believe the fs should be sandboxed, though.

[joskuijpers]

Hmm. The whole sphere js stuff just feels so fragile.

I do really like the idea of having ~/ as the directory with user stuff (App Data / Application Support), because it is writable. Remember that for OSX, I can't/not supposed to write to the .app folder, which is where the engine and all code resides. So if I don't map ~/ to the app data folder, it means we need to expose the user name, the operating system, operating system version, and all that kind of horrible stuff to the game so they can figure it out themselves...

Did you EVER write something outside your app directory with a sphere game? Please give me 2 good examples. :wink:

[FlyingJester]

We don't need to expose anything sys-fs-based directly to the game maker. They only need to be able to traverse fs freely, and know that if the player inputs a valid path it will work in the engine.

Cases of using more than just the current game's directory would be to import images (for instance, as a banner or avatar), importing data from other games (like converting saves from a prequel), reading music from the user's music collection as background music, etc.

It's not that these are things that would be outright promoted. They are, of course, unlikely. But I don't believe they should be prohibited, either. Not knowing what this freedom is really good for is not a good reason to take it away.

[joskuijpers]

Hmm you are right, there.

But I still want ~/ ghehe, or something alike. Maybe @/ and @~/? @ is forbidden anyways. Then a query is resolved, unless it starts with @/ or @~/, which means they are real paths but relative to game root and app data root. If they start with / or ~/, it is an actual real path, so real paths can still be used.

Then if a query starts with @/ it is replac, otherwise it is assumed to be a real path. Then we can still do require("link"), which is resolved because there is no file 'link' at ./. If you want to place a file in user-config, you use new File("@~/config/name")

ANOTHER, cleaner, possibility, is to use System.resolve for resolving stuff in require(). and making the function public. Now if the game-maker wants to user the resolver to find the file 'cache', it can use new RawFile(System.resolve('cache')). (Names are not set in stone). This is kinda what is done in Cocoa: You can just pass any path, or use -[NSBundle pathOfResourceWithName:extension:]. This would thus allow real paths, and also paths to resolve. If resolved, / and ~/ point to game root and app data, if not resolved, they are actually file system paths. Another advantage of this, is that we could generalize ALL paths in the game system to Unix-style paths with slash-forwards. :smile:

[joskuijpers]

The interesting parts of the module system in node.js are at https://github.com/joyent/node/blob/master/lib/module.js, for the ones interested. I am currently trying to implement some. If I implement this, I will need to revise my current JS library to make natives use NativeModules and bindings. You can look at, for example, the os.js, to see how they do bindings to native functions. They load the native module equivalent and assign the methods. We can then minimize the number of functions in native, and the number of function calls.

[joskuijpers]

So today I started implementing my module system. It took some time to work it out. I used many concepts from node.js, which I also had to figure out as they have a much more complicated setup (they have native modules, which are compiled at compile-time and delivered in the binary. I did not do this). You can see the andromeda.js file here: https://github.com/joskuijpers/Andromeda/blob/45419a7bd8c70dd373bc860a817690ee279cb8af/Andromeda/Resources/JavaScript/andromeda.js It has the Module class and supplies require(). It has encapsulation for modules, etc. And I tested it ofcourse: works as a charm.

Now I need to clean things up, heavily. And then I will move ObjC code to JS and minify my ObjC stuff.

Even though I call the native runInThisContext(), I think all of you can do that. @FlyingJester can use the same stuff (V8 Context, Scropt, Compiler, Run), and for @Radnen: it is essentially executing the code in the current JavaScript context and taking its return value.

Loading a script as (function(process) {}); returns a Function object which can then be called upon.

I hope you guys understand some of the code.

[fatcerberus]

I've begun to implement a proper CommonJS module system in minisphere. A prototype implementation is in minisphere 3.0, but is rather limited and can only load designated modules (those included with the engine and those in ~sgm/lib/); arbitrary scripts in the game package must still be pulled in using RequireScript().

Ideally the system will work more like node.js: Designated modules would be loaded without an extension like they are now, e.g. require('link)while arbitrary scripts would be specified by their full path, e.g.require('battle/unit.js')`. This is planned for minisphere 4.0.

Despite my initial misgivings, the module system is a good thing to have. The problem with RequireScript is that it's always global. var declarations at the top of the file become global variables, and if you need them to be local you need to wrap the whole thing in a self-executing function. CommonJS require does the isolation for you automatically, avoiding the need for a function wrapper and getting rid of an indent level.

If you still want global access to a module like you'd have with RequireScript, it's easy enough to import it globally (not recommended to do this inside of a module, though):

global.link = require(`link`);

I've found that the real strength, though, is your dependencies can be specified as locally as necessary and you get lazy loading for free. For example if the function below is never called, the module doesn't need to be loaded, potentially saving memory and minimizing initial load times:

function getEatenByThePig()
{
    var bfp = require('bigFatPig');
    bfp.devour(me);
}

The first request for a given module is cached and future require calls for that same module will return the cached one, avoiding multiple loading. So it works just like RequireScript except you control the visibility.

The only real downside is some additional verbosity for constructor calls:

var audio = require('audio');
var sound = new audio.Sound('sounds/munch.wav');
var mixer = new audio.Mixer(44100, 16, 2);
sound.play(mixer);

But given all the benefits mentioned above, that's a small price to pay in the grand scheme of things.

[joskuijpers]

Hi!

Nice.

Quick correction: in Node, you load local files when you start with a relatie path. That is, you start with ./ or ../. Not sure about x/. But you for sure do not need to add an extension because that actual file might not exist. When you load a module, say 'bluebird' it looks up the bluebird folder in /node_modules and sees in package.json what the main file is. It loads that file. If there is no such file, it looks at the index.js in the folder. If there is no such file it looks at bluebird.js as file instead of bluebird as folder.

Greetings,

On Apr 23, 2016, at 11:10 PM, Bruce Pascoe notifications@github.com wrote:

I've begun to implement a proper CommonJS module system in minisphere. A prototype implementation is in minisphere 3.0, but is rather limited and can only load designated modules (those included with the engine and those in ~sgm/lib/); arbitrary scripts in the game package must still be pulled in using RequireScript().

Ideally the system will work more like node.js: Designated modules would be loaded without an extension like they are now, e.g. require('link)while arbitrary scripts would be specified by their full path, e.g.require('battle/unit.js')`. This is planned for minisphere 4.0.

Despite my initial misgivings, the module system is a good thing to have. The problem with RequireScript is that it's always global. var declarations at the top of the file become global variables, and if you need them to be local you need to wrap the whole thing in a self-executing function. CommonJS require does the isolation for you automatically, avoiding the need for a function wrapper and getting rid of an indent level.

If you still want global access to a module like you'd have with RequireScript, it's easy enough to import it globally (not recommended to do this inside of a module, though):

global.link = require(link); I've found that the real strength, though, is your dependencies can be specified as locally as necessary and you get lazy loading for free. For example if the function below is never called, the module doesn't need to be loaded, potentially saving memory and minimizing initial load times:

function getEatenByThePig() { var bfp = require('bigFatPig'); bfp.devour(me); } The first request for a given module is cached and future require calls for that same module will return the cached one, avoiding multiple loading. So it works just like RequireScript except you control the visibility.

The only real downside is some additional verbosity for constructor calls:

var audio = require('audio'); var sound = new audio.Sound('sounds/munch.wav`); var mixer = new audio.Mixer(44100, 16, 2); sound.play(mixer); But given all the benefits mentioned above, that's a small price to pay in the grand scheme of things.

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub

[fatcerberus]

I know how Node's module loader works, I've looked at the algorithm specification more than a few times. :)

I think it's too complicated though, what I described above was to simplify the module loader and make it more deterministic and predictable. Sphere programmers are used to working with filenames directly, so it wouldn't be outlandish to do require('file.js') I don't think.