vitaly-t / decomment

:hammer: Removes comments from JSON, JavaScript, CSS, HTML, etc.
75 stars 21 forks source link

Error happens when decomment parses some regular expressions #15

Open DIYgod opened 6 years ago

DIYgod commented 6 years ago

A minimal demo:

var decomment = require('decomment');
var code = 'if(true){}/\\d/.test(\'\')';
decomment(code);

Error log:

Error: Line 1: Unexpected token ILLEGAL
    at ErrorHandler.constructError (/Users/DIYgod/Code/test/node_modules/esprima/dist/esprima.js:5004:22)
    at ErrorHandler.createError (/Users/DIYgod/Code/test/node_modules/esprima/dist/esprima.js:5020:27)
    at ErrorHandler.throwError (/Users/DIYgod/Code/test/node_modules/esprima/dist/esprima.js:5027:21)
    at Scanner.throwUnexpectedToken (/Users/DIYgod/Code/test/node_modules/esprima/dist/esprima.js:5155:35)
    at Scanner.getComplexIdentifier (/Users/DIYgod/Code/test/node_modules/esprima/dist/esprima.js:5488:23)
    at Scanner.scanIdentifier (/Users/DIYgod/Code/test/node_modules/esprima/dist/esprima.js:5556:67)
    at Scanner.lex (/Users/DIYgod/Code/test/node_modules/esprima/dist/esprima.js:6223:26)
    at Tokenizer.getNextToken (/Users/DIYgod/Code/test/node_modules/esprima/dist/esprima.js:6666:84)
    at Object.tokenize (/Users/DIYgod/Code/test/node_modules/esprima/dist/esprima.js:154:36)
    at Object.parseRegEx (/Users/DIYgod/Code/test/node_modules/decomment/lib/utils.js:37:13)
vitaly-t commented 6 years ago

When Esprima doesn’t like your code, it must an invalid JavaScript ;)

DIYgod commented 6 years ago

The compressed source code is like

if(true){}/\d/.test('');

Then gulp-strip-comments parses my code with toString function to a string like 'if(true){}/\\d/.test(\'\')'

Should I make an issue to gulp-strip-comments?

vitaly-t commented 6 years ago

I’m not sure, I will test it later though ;)

vitaly-t commented 6 years ago

Here's the test I've done, and it all looks fine:

const decomment = require('decomment');

const str = 'if(true){}/\d/.test(\'\');';

const s = decomment(str);

console.log(s);
//=> if(true){}/d/.test('');

Have you tried gulp-decomment?

DIYgod commented 6 years ago

Sorry for my late reply.

The error happends in /\\d/, not /\d/

vitaly-t commented 6 years ago

You should log/chase it against the Esprima engine as the one that fails to process that piece of JavaScript.

There is nothing that can be done about it within this library.

vitaly-t commented 6 years ago

There, I've opened an issue: Fails on a compressed piece of code.

DIYgod commented 6 years ago

Thanks!

vitaly-t commented 6 years ago

The author of Esprima claims he fixed it, but I've just released an update, and I can still reproduce the issue :(

vitaly-t commented 3 years ago

Esprima hasn't been published for 3 years, and now looks a bit abandoned.

I presently do not have time for rewriting this library to use a different parser.