MDC grammar contains invalid Oniguruma token `[^]`

[slevithan]

This line in the MDC grammar is an invalid Oniguruma pattern. The behavior for TextMate grammars when an invalid pattern is encountered (dutifully reproduced by vscode-textmate) is to silently fail, causing the regex to never match. So whatever it's intended to do is not happening.

The error in this pattern is the use of [^]]. From context, it's easy to figure out that the author meant for this to mean "match any character except ]"). However, that's not what it's doing. There are several different ways that different regex engines interpret this:

• Most regex engines would say that, because ] is the first token within the (negated) character class, it is a literal ]. So, this would be a char class that matches any char except ]. This is what the author intended.
• JavaScript in Unicode-aware mode (with flag u or v) would say this is a negated empty char class followed by an unescaped ], with the unescaped ] being an error.
• JavaScript in Unicode-unaware mode (without flag u or v) would say this is a negated empty char class followed by a literal ]. So, this would match any char plus a following ].
• Oniguruma uses the same interpretation as JavaScript in Unicode-unaware more (that this is a negated empty char class followed by a literal ]) except that, in Oniguruma, an empty char class is an error.

To fix this, the [^]] should be replaced with [^\\]] (escaped backslash since it's in a JS string). It will then run in Oniguruma.

[antfu]

Hey @slevithan, thanks a lot for sorting them out! It must a lot of work, and it would benefit the ecosystem a lot!

However, this repo does not control those grammars; it only syncs and normalizes them. Here is the list of each grammar's source repo, would you be interested in cross-post in the corresponding repos as well, so the original authors/maintainers should be aware of them and fix. If their maintenance is stale and then we could consider how we patch them here as workarounds.

[slevithan]

However, this repo does not control those grammars; it only syncs and normalizes them. Here is the list of each grammar's source repo

Found the list here. No problem, and thanks for the details. Reported this upstream here, and also reported the other recent issues I filed here upstream.

thanks a lot for sorting them out! It must a lot of work, and it would benefit the ecosystem a lot!

Thanks, but this actually doesn’t take much effort 😊 since I’m reporting rules I’ve already coded into oniguruma-to-es. Understanding the innumerable cross-flavor regex differences out there has long been an interest for me, and is something I’ve extensively researched in the past e.g. for Regular Expressions Cookbook, and of course when recently developing Oniguruma-To-ES.