Harmony support

[ozanmakes]

I'm trying to use the excellent livenode with a project which uses ES6 features introduced in node v11, but unfortunately UglifyJS's parser chokes when it encounters a let statement or a generator function (function *() { yield true }).

[mgol]

There can be only one node_modules/uglifyjs

That's definitely one of the issues with npm@3.x

npm 3 only flattens what can be flattened, if you have two packages in dependencies that depend on conflicting versions of a specific package, this package won't get flattened. npm@3 separates the logical dependency tree from the physical one - the directory structure. The logical tree is the same as the directory tree in npm@2 but it shouldn't matter that it's no longer the case in npm@3.

[RevanProdigalKnight]

I've been using npm 3.x for a while now for my own projects and I haven't run into any dependency issues.

[Martii]

Yah looks like npm@2.x has one folder too for uglyfy-js when I include it manually as a depth 0 dep and express-minify requests release... the physical folder is the harmony branch and no uglifyjs2 (uglify-js folder) in the express-minify folder.

@fabiosantoscode or anyone... So do you think a name change would help? say "uglify-js-harmony" ? This is purely git retrieval so that is all release would need to observe I think.

[Martii]

@fabiosantoscode Noise time... sorry in advance... that gives me two physical folders with npm@2.x ... one at depth 0 and one under express-minify ... logical (in memory) testing next with instance provided by express-minify package.

[lewispham]

@mishoo Adding lables is only available to contributors. Am I missing something?

[klimashkin]

Hi! Are there any concrete plans about release date? Or maybe month? Chrome 49 already 92%ES6, ff45 - 85% Our project already have build for modern browsers with less babel transformations, but, unfortunately we cant uglify it. (https://github.com/mishoo/UglifyJS2/issues/448#issuecomment-150728325 doesn't help)

[kzc]

@fabiosantoscode > It's not a fork, it's me trying to get npm to download 2 uglifyjs modules under different names. There can be only one node_modules/uglifyjs

It may be easier for you and potential npm users if the harmony branch was forked with a new module name. May I suggest fabiosify?

[klimashkin]

Maybe simple UglifyJS3-alpha1..n :)

On Thursday, January 21, 2016, kzc notifications@github.com wrote:

@fabiosantoscode https://github.com/fabiosantoscode > It's not a fork, it's me trying to get npm to download 2 uglifyjs modules under different names. There can be only one node_modules/uglifyjs

It may be easier for you and potential npm users if the harmony branch was forked with a new module name. May I suggest fabiosify?

— Reply to this email directly or view it on GitHub https://github.com/mishoo/UglifyJS2/issues/448#issuecomment-173818167.

Sincerely yours, Paul Klimashkin.

[avdg]

We don't have anything stable for release yet. While for example v8 (chrome's js engine) had lots of things behind flags for a long time we had yet to create a harmony branch.

Now we do have one we rely on implementors and testers and because es6 is larger than just a few features this may take some time.

But note that PR's are merged pretty quickly and thus implementation are currently the things that block progress for now (well they take some time anyway).

[kzc]

Timely merges aren't really the issue. It's the fact that there's no distinct npm module name and CLI script name for uglifyjs-es6. This limits its use and testing in the wild.

[avdg]

Yeah, an external package would be nice and wouldn't conflict with the current package.

[StephanBijzitter]

Any updates on this? I just bumped into #938

[julkue]

Just got this error when having an arrow function:

Running "uglify:dist" (uglify) task
JS_Parse_Error {
  message: 'Unexpected token: operator (>)',

so it seems to have no update

[sbrl]

@julmot I think you need to use the special harmony branch of UglifyJS2,

[c-f-h]

I'll second that it would be great to have a separate experimental package published on npm with ES6 support.

[PokemonAshLovesMyTurkeyAndILikeYouTwo]

Do you guys know if it's possible to edit one of the files here:

uglify-js\lib

and make it so the uglifying process basically skips template strings?

See my issue, I know ES6 support isn't ready, but I'm only using template strings and just need to find a way to skip that process if possible

Edit: it looks like the error is coming from

function parse($TEXT, options) {

inside the parser.js file

If anyone can try to get a temporary fix so it can skip the template strings that would be great. If I find a fix first, I'll post as well

[PokemonAshLovesMyTurkeyAndILikeYouTwo]

Great News! Template Strings are now working!

WARNING: This is extremely hacky and only use if you are not using tilde keys INSIDE template strings.

Follow these simple steps:

Step 1

Replace the read_string function at /lib/parse.js with:

    var read_string = with_eof_error("Unterminated string constant", function(quote_char){
        var quote = next(), ret = "";   
        for (;;) {
            var ch = next(true, true);
            if (ch == "\\") {
                // read OctalEscapeSequence (XXX: deprecated if "strict mode")
                // https://github.com/mishoo/UglifyJS/issues/178
                var octal_len = 0, first = null;
                ch = read_while(function(ch){
                    if (ch >= "0" && ch <= "7") {
                        if (!first) {
                            first = ch;
                            return ++octal_len;
                        }
                        else if (first <= "3" && octal_len <= 2) return ++octal_len;
                        else if (first >= "4" && octal_len <= 1) return ++octal_len;
                    }
                    return false;
                });
                if (octal_len > 0) ch = String.fromCharCode(parseInt(ch, 8));
                else ch = read_escaped_char(true);
            }
            else if (ch == quote) break;
            ret += ch;
        }

        if(quote_char.match(/`/)){
              var tok = token("string", ret);
              tok.quote='MAKETEMPLATESTRINGSGREATAGAIN';
              tok.value = tok.value.replace(/\s/g, "") // Remove the extra spaces leftover from our hack
        }else{
        var tok = token("string", ret);
        tok.quote = quote_char;
        }
        return tok;
    });

Step 2

Replace the switch statement inside function next_token in /lib/parse.js to:

        switch (code) {
          case 34: case 39: return read_string(ch);
          case 96: return read_string(ch); // Handle Template Strings
          case 46: return handle_dot();
          case 47: return handle_slash();
        }

Since tilde key has a keyCode of 96

Step 3

Now in /lib/output.js replace the function quote_single() with:

        function quote_single(ev) {
                if(ev == 'MAKETEMPLATESTRINGSGREATAGAIN'){
                        return "`" + str.replace(/\n/g, "\\'") + "`";
                }else{
                        return "'" + str.replace(/\x27/g, "\\'") + "'";
                }
        }

And.. right below that, there is a switch statement, replace that to:

        switch (options.quote_style) {
          case 1:
            return quote_single();
          case 2:
            return quote_double();
          case 3:
            return quote == "'" ? quote_single() : quote_double();
          default:
            return dq > sq ? quote_single(quote) : quote_double();
        }

(we are now passing the quote property we assigned earlier so we can check if TEMPLATESTRINGSAREGREATAGAIN inside the quote_single function above.

Boom. Done.

Proof:

Before:

After:

Have a wonderful day everyone, enjoy!

[rvanvelzen]

Great News! Template Strings are now working!

They've been working for ages in the harmony branch. Please don't misinform other people by posting junk fixes.

[PokemonAshLovesMyTurkeyAndILikeYouTwo]

They've been working for ages in the harmony branch. Please don't misinform other people by posting junk fixes.

Last time I checked 'working on' doesn't mean it's accessible. My fix works fine for me, especially since I'm ONLY using template strings and need my game to be compiled now. I don't got time to wait for ages, nor hear negative & condescending things from unappreciated people like yourself.

[rr-]

Or you can just target harmony branch in your package.json like this and not have to maintain some weird local fixes.

[mgol]

@PokemonAshLovesMyTurkeyAndILikeYouTwo Your hack is incorrect, you can't just remove white spaces from inside template strings, this changes what the code does.

[PokemonAshLovesMyTurkeyAndILikeYouTwo]

@mgol

Hack may be 'incorrect', but I have already compiled tons of scripts using it with no issues.

@rr- That's a good idea and way better than my solution. Thank you for that. I didn't know you could use harmony / git like that. I wish I knew, I would of just did that.

[mgol]

You may have as well compiled tons of scripts where you replace all occurrences of the string 'all' with 'none' but if it worked for you this still wouldn't prove this patch is right.

[avdg]

I'm currently running some test262 tests on harmony to see how far we are progressed by now. There are quite some errors and the logs are huge (while the test is still running) but I hope to extract some useful information from it.

There's been a lot of work done on the harmony branch, and we might want to thank those who contributed to or made it possible to get the code base we have today.

[avdg]

This is the result I have from test262 thrown on the harmony branch of UglifyJS. It also contains the run environment data, but lacks detailed error messages.

https://gist.github.com/avdg/f34c991cf306a62ca67cc6622bc12c96 outdated

I hope it gives a nice overview on what still has to be done (I have no time yet to look it in details myself).

Edit: please note we don't have to fix all issues, we just have to fix the most important ones. Besides, not all issues are caused by UglifyJS alone since the tests still needs to be executed in a js environment like node. We do have to select the most important problems we can fix. Once we got over there, we will put ourselves in a sprint for an eventual release.

[fabiosantoscode]

I was looking at the failures, and it seems a bit weird that things like Promise fail. Promise is not a syntax feature, it's a standard library addition, so I wouldn't expect any failures in uglifyjs.

[avdg]

Promise errors aren't caused by UglifyJS, they are caused by node because the tests still have to be executed. I'm still working to get better information in the markup files to clarify that and will update the gist partially later. All that stuff is still in development.

Tests definitely caused by UglifyJS will get a rubberstamp telling that the error is caused by UglifyJS soon. For now, only the directories are listing how many errors are caused by UglifyJS.

Running the test262 suite takes almost 3 hours on my computer, converting these logs to markup format takes less than a second, but the program will be adjusted to the needs.

[avdg]

I've updated the gist with fixes I've already made.

[avdg]

Updated the gist even more now, some tests have their cause determined. Note that the results did ran on node 6.0.0 without the harmony flag. I should have done it with the flag on.

[avdg]

Uh wrong setup :-) I wanted to post some gist results but they were invalid.

[Martii]

@avdg Hehe... one of those days/weeks. ;) I appreciate your testing... steers me in some alternate directions/thoughts too. Keep up the great work... this applies to everyone too. :)

[avdg]

Yeap, tests are fixed for now :-)

https://gist.github.com/avdg/85ddc1e7ed2c6f32987446620fa958f2 - running on node with harmony flag

https://gist.github.com/avdg/43002267f959210a69aa3ec0f21ed751 - running on node with harmony flag

[willin]

es7 async

[avdg]

Added more types of errors including parse errors to the results overview. Pretty sure they are useful.

[KingScooty]

Potentially daft question, but i'm just trying to get up to speed with this thread. How do i get Webpack 2 using this Uglify harmony branch?

[christophehurpeau]

@KingScooty just add this as a dependency: "uglify-js": "github:mishoo/UglifyJS2#harmony"

[KingScooty]

@christophehurpeau Nice! Will webpack automatically pick that up from the package json?

[juodumas]

Oops, didn't know this was already answered (e-mails seem to be delayed). @KingScooty: just do this and see for yourself: $ npm i github:mishoo/UglifyJS2#harmony --save-dev $ webpack -p

[KingScooty]

Just works too! Thought there'd be more to it than that! Love it! Cheers guys!

[avdg]

Fresh logs: https://gist.github.com/avdg/d5c16fb6d12a174da8ea8c5b92bef067 uh, master branch used, not sure what caused the misconfigure.

Seems I still have yield parsing bug afterwards due to wrong filter (not all tests such tests are labeled yield) (edit: need to run another test as it was run without the yield fixing patch)

Real fresh logs: https://gist.github.com/avdg/4eafd595701da85a7a7dc2251d8978a0

[iceprosurface]

so arrow function syntax still cannot be used? if could ，i vote to '=>'.

[kevincox]

Yeah. I'm not going to like the only reason I am transpiling one project to es5 is so that I can minify it with uglify. It would be really nice to get basic es6 support working.

[fabiosantoscode]

Arrow functions and a lot of other es6 features are supported by the harmony branch. To install, use

npm i mishoo/UglifyJS2#harmony

[avdg]

test262 results are just an indication. The tests itself may be over the top and uglify may ship with failures on test262 depending on the context of these errors (tests are just designed to fail despite the lack of real usage in the real world).

[avdg]

Also I like to note that the test262 suite is in active development. So it might be possible that we might fail more and still have improved on it. Also there are small features expected to be added in ecmascript 2016, but they are very small (** / **= operator and Array.prototype.includes).

[jprichardson]

@kevincox check out Google Closure Compiler. It officially supports es6 since October 2015 and compress js better. The only cons is it requires java.

@egoroof it's my understanding that it doesn't output ES6 though. That's a problem, especially if you want to use native generator support. So by that metric, Uglify supports ES6 as well since you can preprocess with Babel. Please correct me if I'm wrong.

[klimashkin]

@egoroof Google Closure Compiler doesn't compress ES6. First it transforms it to ES5 with traceur-compiler and then compress as usual. Try it https://closure-compiler.appspot.com/home#code%3Dfunction*%2520rr%2520()%2520%257B%250A%2520%2520const%2520r%2520%253D%2520yield%2520nn()%253B%250A%257D%250A

[avdg]

Here are the correct fresh logs as I didn't notice that testing was done on an older commit: https://gist.github.com/avdg/4eafd595701da85a7a7dc2251d8978a0 outdated

Current summary:

• Seems yield * expression was allowed which is my bad.
• Most failures seems to be spread/expand operator related, which is going to be the next priority.

[egoroof]

@jprichardson and @klimashkin yes, you are correct. Sorry for misleading.

[avdg]

ES 2017 / ES 7 is out now, and since it's easy to implement the **= and ** operators, I'm doing them now. This while chrome doesn't seem to support ** operators yet. The other important change is Array.prototype.includes, but support for that will automatically be included in uglifyjs once it is supported the execution host (which is mostly node, unless ported to the browser).

On the background I am trying to document on everything related to ([{ ... some content that may be , separated... }]) because that seems to be complex and easy to add duplicated code.

Maybe I have to take a look at computed properties as well as they don't seem to behave good enough in the test262 tests.

Support for spread in array literals is already implemented in #1125 but is still not merged yet.