Make this more versatile via a simple plugin system

[silkentrance]

I wonder why you take on the burden of having to maintain the file extensions at all?

How about implementing a simple plugin system that would allow users to integrate their own "experimental" languages and other languages that are currently not supported.

I am currently working on a similar solution and I would be happy to stop working on that, provided that node-interpret would allow me to plug in my own language without having to undergo a request/deny/clone/maintain my own clone process :-)

I proposed my prototypical solution here: PR gruntjs/grunt#1269. I also begun setting up a repository for a pluggable library, very similar to yours over at vibejs/vibejs-dialector

The gist of both is that there exist a multitude of plugins, e.g.

node-interpret-coffee node-interpret-typescript node-interpret-coco ...

which in turn will take care of registering the languages with node and each of them implementing a simple interface, which will allow users to

a. switch between language versions easily, e.g. adding a dependency towards node-interpret-coffee@1.8.0 would allow them to use version 1.8.0 of CoffeeScript whereas a dependency towards node-interpret-coffee@1.3.3 would let them use version 1.3.3 of that language b. integrate custom dialects / experimental versions of their own c. never having to undergo the request/deny/... process ever again

Please let me know what you think of that.

[tkellen]

The extensions are listed here because they are used by command line tools which would like to consume configuration files in any format. For example, gulp-cli. When tools powered by Liftoff see a configuration file with an extension name that exists in this listing, they know how to register the module loader so the file can be read transparently (the end user has to have the correct module available locally, though).

Node has a plugin system natively--it's called require.extensions, and there is already a great ecosystem out there:

• cjsx: https://github.com/SimonDegraeve/node-cjsx/blob/master/index.js
• coco: https://github.com/satyr/coco/blob/master/lib/node.js#L28-L41
• coffeescript: https://github.com/jashkenas/coffeescript/blob/master/lib/coffee-script/register.js
• csv: https://github.com/olalonde/require-csv/blob/master/index.js
• ini: https://github.com/olalonde/require-ini/blob/master/index.js
• json5: https://github.com/aseemk/json5/blob/master/lib/require.js

.... etc

I'm confused what you're looking for? If you have a new javascript variant you are working on that has a module which makes it possible to require those files in node, I would be more than happy to include it here!

[silkentrance]

@tkellen to refer to my initial post and to the very first sentence therein:

I wonder why you take on the burden of having to maintain the file extensions at all?

following up with

How about implementing a simple plugin system ...

that would leverage that burden from you and instead impose it onto the community to provide the plugins required for node-rechoir/node-interpret to find and resolve the languages/compilers and register them with node/require.

Simple usecase:

1. I am currently implementing version 0.0.1-pre-alpha of my own language
2. I request from you that you support that language
3. You deny or simply do not respond ASAP
4. I cannot use my tools and would have to fork either node-interpet or all of the tools that use node-rechoir/node-interpret in order to use these tools

Instead I would like to

1. create my very own language
2. create a stub plugin for that located under for example /usr/lib/node_modules/
3. simply use the tools I am comfortable with and with the experimental version of my own language

As you can see, the second path would not require any involvement of you as, so to say, the gate keeper of what gets supported and what not.

I,. for my part, would be much more comfortable with such a solution and you should be too, as this makes any requests towards you on what to support in the future simply go away, freeing up your time for more productive work.

[tkellen]

Hmm. Step #3 would never happen. If you have a loader for your new language, I'd be thrilled to include it! Just submit a PR and we're off to the races.

The community has already provided the plugins, and node has already provided the loading mechanism. The only thing happening here is aggregating the work of the community into something that can be consumed.

With respect to time, to be totally honest, I've spent more time responding to this issue than I did creating this library or merging and publishing any of the PRs that have been made against it in the eight months or so that it's been around. I'm not saying that as a complaint, by the way, I appreciate your enthusiasm!

The burden you perceive here is non-existent. Finally, if you're going to be creating scads of programming languages and need to update this like, daily, I'd be happy to give you commit bit and npm push access to this project.

[silkentrance]

@tkellen just what I thought it would turn out. no way. thanks for implementing a closed solution and make it stick. I will continue with vibejs-dialector and see if the tools/libraries will support this as an alternative, if not, i will maintain my own forks of these tools/libraries.

[yocontra]

gulp has --require which lets you load up any language you want, so you don't need to send a PR if you have something custom you are doing locally.

[silkentrance]

@tkellen how, by any chance, do you handle different versions of the languages? Say for example I have a version 1.0.0 of my own language that is incompatible with version 1.1.0 of the same language.

How would I be able to use version 1.1.0 in one project and with the same tools and a different project would still be using version 1.0.0 of the same language.

And both of these two versions would require different ways to register them with node?

As for gulp --require, yes, that might be a viable option, however, as with mocha, this will still leave me with tool specific semantics. I'd rather have a tool indifferent semantic such as adding a dependency toward for example node-interpret-stylishlanguageprototype@0.0.1 to my package.json instead of having to instruct every tool I use which version of a specific language I would like to use.

[tkellen]

If you're referring to rechoir, it defers to whatever version of the module is found locally. You can use any version of the module you want, but you do have to install it.

[yocontra]

@silkentrance interpret doesn't care what version you use as long as you expose the same interface was what it supports. If you put coffee-script@1.7 in your package.json of your project it will load that, or if you use npm link to link a local version in another project it will use that.

[silkentrance]

Considering the hard coded extensions, I find it difficult to think of node-rechoir or any dependent tools/libraries to support different versions of the very same language.

Say, version 1.0.0 of my language will simply register itself, as is the case with coffee-script. Whereas version 1.1.0 will use a /register module. Now, version 1.2.0 make use of yet another registration mechanism, lets call it ngenreg, while still being backwards compatible language wise to version 1.0.0 and thus having the same extension, say '.fancy'.

How would you cope with that?

In my world, aka the simple plugin system, individual versions of the same plugin would implement different semantics on how to register the compiler/extension with node.

[yocontra]

Is this an actual problem that exists or are we trying to solve theoretical use cases that may possibly maybe happen if somebody did something a certain way at some undetermined date in the future? Creating another silo for plugins should only be done when absolutely necessary

[silkentrance]

Now, coffee-script versions 1.3.3 and up until some version where the /register module was introduced, will register themselves. Whereas at least version 1.8.x onwards will require calling the /register module.

Apart from that, the plugins that I am talking about are mainly composed of a descriptor which is part of the package.json and a simple register.js module that will implement the logic that is now part of node-rechoir. While the logic may be similar for a multitude of dialects now, future dialects/dsls might require different loading mechanism. Something, I reckon, which is already part of node-interpret as part of the special cases in the register export.

Having such special cases in node-interpret and also having to maintain them for every language/dsl seem to me like a bit of pain in the ...

[tkellen]

Thanks for taking the time to explain your line of thinking further, Carsten.

I think the solution you are proposing add needless complexity to what is currently a very simple and completely functioning system. I can measure the time I've spent on this library in hours, on one hand. It requires essentially no maintenance. This is because all of the functionality surrounding module loading lives where it belongs: in the modules themselves. All interpret does is point to them.

I think it's fair to say this library is battle tested, having been downloaded several million times since its release. In that time, there has not been a single report of someone running into the problems you're trying to solve.

Yes, the scenarios you are describing could happen, but in reality, they haven't. Even if they did, I think your solution is the wrong one. Building, publishing, evangelizing and maintaining a standalone plugin for every transpile-to-js module in the ecosystem seems like madness to me. Doubly so when they aren't needed.

The best shared solution I can think of at this point is one (or both) of these ideas:

1. I will happily merge and publish a patch that adds support to interpret for your new programming language.
2. I will happily merge and publish a patch to rechoir that supports providing your own interpret object.

[phated]

I agree with @tkellen and as @contra pointed out, there's already a way around interpret in gulp.

[silkentrance]

@tkellen Well, then we will try going for a patch for rechoir :-)

One more thing, though. What if I were to implement a variant of CoffeeScript that would not only compile .coffee/.litcoffee but would also compile, say, .guava? Currently, when coming across .coffee/.litcoffee, rechoir would try loadinjg coffee-script instead of the "new" guava variant.

But I guess, that this could be solved with a patch to rechoir as well... I will have a look into this.

[tkellen]

Sounds good! Closing this--I will look for your PR :D