Detect es6 automatically for .js endings (or treat every file as es6)

[aurimus]

When I include a library that uses es6, it usually has file extentions of .js and not .es6, sprockets does not detect the type automatically therefore it's not compiled.

I would like to be able to either set somewhere in configuration for it to treat every file as es6 or so it detect it automatically.

• Sprockets version 4.0.0beta4
• Ruby version 2.3

[schneems]

Is it es6.js or just plain .js? Can you give me an example? If it's es6.js we could add a mimetype for that extension.

[aurimus]

no, just plain .js .. you have to look inside the file to detect it, but setting it in settings would be ok

[schneems]

We can't run the babel processor against every JS file. It will take too long. I do think it makes sense to allow for using an ES6 file that doesn't have a .es6 file extension.

We need some way to explicitly say "this is a ES6 file, compile it with babel" via the sprockets interface.

Here's how we implement es6 support now

  require 'sprockets/babel_processor'
  register_mime_type 'application/ecmascript-6', extensions: ['.es6'], charset: :unicode
  register_transformer 'application/ecmascript-6', 'application/javascript', BabelProcessor
  register_preprocessor 'application/ecmascript-6', DirectiveProcessor.new(comments: ["//", ["/*", "*/"]])

Where the context would be Rails.application.assets (or another instance of Sprockets::Environment).

The way that sprockets works is you define a mime type, and then you tell it what file extensions are that mime type in this case files ending in es6 are mime type application/ecmascript-6. We then tell it what that mime type transforms into, in this case 'application/ecmascript-6' transforms into 'application/javascript' using the BabelProcessor.

One possible solution could be to add a .js extension to the es6 mime types

  register_mime_type 'application/ecmascript-6', extensions: ['.es6', '.js'], charset: :unicode

However this doesn't work out of the box. I'm assuming it has something to do with the problem that .js is already of type application/javascript so trying to convert it to that isn't needed, so the processing step is skipped.

Another method i tried:

I tried making an es6 file and then using a //=require from that file to import an es6 with a .js extension. This does not work because while the directive //= require is processed before we apply the BabelProcessor. We don't concatenate the source from the require directive into the original file until after the BableProcessor is run. This happens when the Bundle processor gets executed.

The short version of this is: This is a really hard problem with the way that sprockets is built. It may be easier to compile the 3rd party es6 files into JS using something like webpacker and then link to the compiled files as simple JS.

[aurimus]

Wouldn't it be possible to just try to compile everything? ES6 is backwards compatable to older JS versions so every file that's an old JS would simply compile to itself.

[schneems]

Shouldn’t it be possible? Yes. How do I do it? No clue.

[aurimus]

There must be someone who does .. compile every file, seems simple enough

[schneems]

I’m the maintainer. If it’s so simple please send me a PR.

[bbugh]

I would very strongly prefer that all .js is just babelized if the babel-transpiler gem is installed. I don't want to have to deal with reconfiguring the ecosystem of editors, linters, etc. all set up for .js extension, not to mention vendor libraries and so on. Yikes!

@schneems I got pretty close to making it work, maybe with some of your extra domain knowledge we can fix it. With the changes I made, an input .js file using ES6 features like this:

// source: app/assets/javascripts/happy.js
const happy = (msg='I transpiled into ES5!') => {
  console.log(`Wow! I am so happy because ${msg}`)
}

Results in:

// output: http://localhost:3001/assets/application.js
// ...
var happy = function happy() {
  var msg = arguments.length <= 0 || arguments[0] === undefined ? 'I transpiled into ES5!' : arguments[0];

  console.log('Wow! I am so happy because ' + msg);
};
// ... 

The change:

diff --git a/lib/sprockets.rb b/lib/sprockets.rb
index a350c1c7..a75389c4 100644
--- a/lib/sprockets.rb
+++ b/lib/sprockets.rb
@@ -144,9 +144,10 @@ module Sprockets

   # Babel, TheFuture™ is now
   require 'sprockets/babel_processor'
-  register_mime_type 'application/ecmascript-6', extensions: ['.es6'], charset: :unicode
-  register_transformer 'application/ecmascript-6', 'application/javascript', BabelProcessor
-  register_preprocessor 'application/ecmascript-6', DirectiveProcessor.new(comments: ["//", ["/*", "*/"]])
+  register_preprocessor 'application/javascript', BabelProcessor

I used register_preprocessor instead of register_transformer to run the BabelProcessor. This is perhaps not the intent of register_preprocessor but like you alluded to in a previous message, the transformers intention seems to be to transpile one mimetype to another mimetype. In the case of ES6, I would argue that it's really just preprocessing. 😎

There's a breaking change though—I had to comment out the register_bundle_metadata_reducer SourceMap line for the javascript due to a nil error. Overall it was a very small and simple change. Hopefully we don't have to do much more to make it work. I will be very pleased if we can get this working without .es6. Principle of least surprise and all that.

I'm going to look into it some more, but on the chance that you know how to work around this it might save some time.

The full changeset is here: https://github.com/rails/sprockets/compare/master...bbugh:transpile-dotjs

[simonhaenisch]

We can't run the babel processor against every JS file. It will take too long.

But I only write ES6+ code... this would only be an issue with vendor files i guess?

[LucasArruda]

Could we get an option to parse .js as ES6? The fact that I have to rename fields based on ES6 is really making adopting sprockets 4 bundled with uglifier 4 a pain.

It seems to be that uglifier 3 simply ignores ES6 fields with .js extension, so they don't compress for me. Which is ok in my case, because we have webpack before sprockets bundling stuff.

But we eventually want to switch to web packer and thus we need this capability of rails handling ES6/JS seamlessly.

Eventually they will all be ES6, but while they are not, why not transpile everything?

A setting for that would be very nice. Monkey patching this or adding settings is not working well for me.

I would be happy to take over this https://github.com/rails/sprockets/issues/460#issuecomment-360253697 changes if they are not already working.

[bbugh]

@LucasArruda we switched to webpacker, and haven't done anything else with this on sprockets. Good luck!

[LucasArruda]

@bbugh thanks. So you haven't been able to use ES6 on webpacker unless you change the extension? I did a try with webpacker, but that involved me having to upgrade to new uglifier, which conflicts with ES6 code and require me to change extension to .es6. That also did not work for you?

[johnnyshields]

Hi, is there any update on this? Many folks are still using Sprockets, so comments about Webpacker are not relevant.

The approach @bbugh is advocating, perhaps with a settings switch to toggle it, seems like a reasonable one. Would the Sprockets maintainers accept a PR for it?