Duktape fails to parse a regexp accepted by chrome

kinnison commented 8 years ago

Embedded in some JS which deals with CSS, we have encountered the following regular expression:

/:((?:[\w\u00c0-\uFFFF_-]|\\.)+)(?:\((['"]*)((?:\([^\)]+\)|[^\2\(\)]*)+)\2\))?/

Chrome accepts this happily, but duktape complains of an invalid decimal escape.

This prevents some sites from loading their javascript frameworks.

Any help would be gratefully received.

svaarala commented 8 years ago

This is one of those real world regexp leniency issues: the issue is probably [^\2] which is not allowed in standard Ecmascript:

DecimalEscape \0 evaluates to a NUL character, while all non-zero decimal escapes evaluate to an integer (E5.1 Section 15.10.2.11).
ClassEscape evaluates the decimal escape and is required to SyntaxError if the result is not a character (E5.1 Section 15.10.2.19). In other words, anything other than \0 evaluates to an integer which causes a SyntaxError inside a class.

Longer term I don't think it's feasible for Duktape's regexp engine to simultaneously be low footprint and support non-standard regexp idioms, so the two alternatives are:

Provide a separate regexp engine with a larger footprint but better performance and better support for extended, non-standard regexps.
Allow hooking an external regexp engine for applications where regexp behavior is especially important (like NetSurf).

I'm planning to start with the second approach, as soon as I get some time to work on it :-)

fatcerberus commented 8 years ago

With the number of real world regexps I've seen Duktape reject, it actually amazes me that something like NetSurf is feasible. It makes me think the leniency issue isn't as big a deal as it seems at first.

danielsilverstone-ct commented 8 years ago

@fatcerberus As it stands, our efforts are not resulting in something compatible with what's out there; but we're working hard to make it go. As @svaarala builds more and more possibility into duktape, so our efforts go further :-)

jfahrenkrug commented 8 years ago

I was about to open a new issue, but I think it's better to just comment on this one.

I have a project built with WebPack that uses the XRegExp npm package. Duktape had problems with these two regular expressions:

/\[(\^?)]/

and

/(\()(?!\?)|\\([1-9]\d*)|\\[\s\S]|\[(?:[^\\\]]|\\[\s\S])*]/g

In both cases the issue is that the regex wants to match a literal ], but it doesn't escape it. So technically the regex is invalid.

When I change the first one to

 /\[(\^?)\]/

and the second one to

/(\()(?!\?)|\\([1-9]\d*)|\\[\s\S]|\[(?:[^\\\]]|\\[\s\S])*\]/g

Duktape is happy :)

svaarala commented 8 years ago

@jfahrenkrug Support for literal curlies was added a while back - I could look into adding support for literal brackets too. They don't appear in unescaped form as commonly as the curly braces but still pop up from time to time :)

svaarala commented 8 years ago

@jfahrenkrug #871; the [ matching is more complicated because it needs actual lookahead, but accepting a literal closing bracket ] should be trivial.

jfahrenkrug commented 8 years ago

@svaarala That's great news, thank you! Meanwhile I've opened a PR for XRegExp: https://github.com/slevithan/xregexp/pull/141

livrish commented 6 years ago

Hi

when we use the selfDefending on javascript-obfuscator the regex chokes also

function outputMyNumber(showThis) { console.log(showThis); } function hello() { var getNumber = function (b) {return b+ 3; } var thenumber = getNumber(2); outputMyNumber(thenumber); } hello()

obfuscator: { options: { compact: true, controlFlowFlattening: false, deadCodeInjection: false, debugProtection: false, debugProtectionInterval: false, disableConsoleOutput: false, identifierNamesGenerator: 'hexadecimal', log: true, renameGlobals: false, rotateStringArray: true, selfDefending: true, stringArray: true, stringArrayEncoding: 'base64', stringArrayThreshold: 0.75, unicodeEscapeSequence: false, target: 'browser' },

svaarala / duktape

Duktape fails to parse a regexp accepted by chrome #425