Prevent unicode on contextual tokens. Fixes #376

shapesecurity / shift-parser-js

ECMAScript parser that produces a Shift format AST

http://shift-ast.org/parser.html

Apache License 2.0

248 stars 28 forks source link

Prevent unicode on contextual tokens. Fixes #376 #384

Closed Protryon closed 5 years ago

Protryon commented 5 years ago

Fixes issue #376.

bakkot commented 5 years ago

LGTM, but needs tests.

(Also, you'll need to get lint and tests passing; you can run npm run lint and npm run test locally.)

bakkot commented 5 years ago

Then we can get rid of TokenType.ESCAPED_KEYWORD.

That would require adding an additional check (for !token.escaped) every time we consume any of the many keyword tokens. I don't think it's worth it.

michaelficarra commented 5 years ago

Yeah you're probably right. We at least shouldn't mark ESCAPED_KEYWORDs with escaped: false, though.

bakkot commented 5 years ago

This should at least have the cases in #376 as tests.