Long regexes break highlighting in Javascript code

[brookback]

Hi! I'm using this package to syntax highlight .js files. I stumbled upon this today:

const emojiUnicodeRegEx = /(?:[\u2700-\u27bf]|(?:\ud83c[\udde6-\uddff]){2}|[\ud800-\udbff][\udc00-\udfff]|[\u0023-\u0039]\ufe0f?\u20e3|\u3299|\u3297|\u303d|\u3030|\u24c2|\ud83c[\udd70-\udd71]|\ud83c[\udd7e-\udd7f]|\ud83c\udd8e|\ud83c[\udd91-\udd9a]|\ud83c[\udde6-\uddff]|[\ud83c[\ude01\uddff]|\ud83c[\ude01-\ude02]|\ud83c\ude1a|\ud83c\ude2f|[\ud83c[\ude32\ude02]|\ud83c\ude1a|\ud83c\ude2f|\ud83c[\ude32-\ude3a]|[\ud83c[\ude50\ude3a]|\ud83c[\ude50-\ude51]|\u203c|\u2049|[\u25aa-\u25ab]|\u25b6|\u25c0|[\u25fb-\u25fe]|\u00a9|\u00ae|\u2122|\u2139|\ud83c\udc04|[\u2600-\u26FF]|\u2b05|\u2b06|\u2b07|\u2b1b|\u2b1c|\u2b50|\u2b55|\u231a|\u231b|\u2328|\u23cf|[\u23e9-\u23f3]|[\u23f8-\u23fa]|\ud83c\udccf|\u2934|\u2935|[\u2190-\u21ff])/g; /* eslint max-len: 0 */

This long regex breaks highlighting kinda mid way. See that the last /* comment */ is highlighted in orange as well.

Version: 2.81.0 Atom version: 1.23.0

Keep it up!

[gandm]

Unfortunately Atom imposes a maximum limit of 100 tokens an a single line for any grammar. This was done to increase performance. See here. It looks like your line has hit that limit and the grammar will stop looking for any more. I could add code to fix thus issue but have decided not to (see next para for reason). Another package allows this limit to be overridden for all grammars so you could TRY IT.

I believe that Max Brunsfeld's (Github/Atom dev) tree-sitter package will ship soon in Atom. It uses a true AST incremental parser that should solve all these TextMate issues ( poor performance, lack of look ahead, limited tokens, etc) plus providing some very neat extras. I'm sure there will be a blog post for it when it ships.

[brookback]

Gotcha, thanks for highlighting (lol) that. No biggie at all, let's chill for the tree sitter package!