Regex we can't parse

[jgm]

 \((?=(?:[^  \\'"|()`ugeP]*+(?:[ug][0123456789]+|[ugeP](?::(?:[^:'`"\\|()]*+(?:\\.|'[^']*'|`[^`]`|"(?:[^"\\`]*+(?:`[^`]`|\\.)?)*")?)*(?::|(?=[|()]))|\[(?:[^]'`"\\|()]*+(?:\\.|'[^']*'|`[^`]`|"(?:[^"\\`]*+(?:`[^`]`|\\.)?)*")?)*(?:\](?=[|()]))|{(?:[^}'`"\\|()]*+(?:\\.|'[^']*'|`[^`]`|"(?:[^"\\`]*+(?:`[^`]`|\\.)?)*")?)*(?:}(?=[|()]))|<(?:[^>'`"\\|()]*+(?:\\.|'[^']*'|`[^`]`|"(?:[^"\\`]*+(?:`[^`]`|\\.)?)*")?)*(?:>|(?=[|()]))|([^  <>|&;(){}'"`\\])(?:(?:(?!\1)[^'`"\\\1|()])*+(?:\\.|'[^']*'|`[^`]`|"(?:[^"\\`]*+(?:`[^`]`|\\.)?)*")?)*(?:\1|(?=[|()]))|(?=\|))|\\.|'[^']*'|`[^`]`|"(?:[^"\\`]*+(?:`[^`]`|\\.)?)*")?)*(?:\)(?:[^  }<>|&;)]|(?:}+(?:[^*?#^~[  <>|&;()${}'"`\\]|(?=[}$'"`\\]))))|[  |(]|$))

[jgm]

This (from zsh.xml) and an increasing number of syntax definitions use possessive quantifiers ++ and *+. https://stackoverflow.com/questions/4489551/what-is-double-plus-in-regular-expressions

We should support these so we can use the up-to-date syntax definitions.

[jgm]

Wait, we DO supposedly support possessive quantifiers. So maybe that is not the issue here.

[jgm]

More regex parse failures

zsh.xml

\$\((?=\(((?:[^`'"()$]++|\$\{[^`'"(){}$]+\}|\$(?=[^{`'"()])|`[^`]*+`|\((?1)(?:[)]|(?=['"])))++)(?:[)](?=$|[^)])|["']))| 

\((?=(?:[^  \\'"|()`ugeP]*+(?:[ug][0123456789]+|[ugeP](?::(?:[^:'`"\\|()]*+(?:\\.|'[^']*'|`[^`]`|"(?:[^"\\`]*+(?:`[^`]`|\\.)?)*")?)*(?::|(?=[|()]))|\[(?:[^]'`"\\|()]*+(?:\\.|'[^']*'|`[^`]`|"(?:[^"\\`]*+(?:`[^`]`|\\.)?)*")?)*(?:\](?=[|()]))|{(?:[^}'`"\\|()]*+(?:\\.|'[^']*'|`[^`]`|"(?:[^"\\`]*+(?:`[^`]`|\\.)?)*")?)*(?:}(?=[|()]))|<(?:[^>'`"\\|()]*+(?:\\.|'[^']*'|`[^`]`|"(?:[^"\\`]*+(?:`[^`]`|\\.)?)*")?)*(?:>|(?=[|()]))|([^    <>|&;(){}'"`\\])(?:(?:(?!\1)[^'`"\\\1|()])*+(?:\\.|'[^']*'|`[^`]`|"(?:[^"\\`]*+(?:`[^`]`|\\.)?)*")?)*(?:\1|(?=[|()]))|(?=\|))|\\.|'[^']*'|`[^`]`|"(?:[^"\\`]*+(?:`[^`]`|\\.)?)*")?)*(?:\)(?:[^  }<>|&;)]|(?:}+(?:[^*?#^~[   <>|&;()${}'"`\\]|(?=[}$'"`\\]))))|[     |(]|$))

\((?=(?:[^  \\'"|()`ugeP]*+(?:[ug][0123456789]+|[ugeP](?::(?:[^:'`"\\|()]*+(?:\\.|'[^']*'|`[^`]`|"(?:[^"\\`]*+(?:`[^`]`|\\.)?)*")?)*(?::|(?=[|()]))|\[(?:[^]'`"\\|()]*+(?:\\.|'[^']*'|`[^`]`|"(?:[^"\\`]*+(?:`[^`]`|\\.)?)*")?)*(?:\](?=[|()]))|{(?:[^}'`"\\|()]*+(?:\\.|'[^']*'|`[^`]`|"(?:[^"\\`]*+(?:`[^`]`|\\.)?)*")?)*(?:}(?=[|()]))|<(?:[^>'`"\\|()]*+(?:\\.|'[^']*'|`[^`]`|"(?:[^"\\`]*+(?:`[^`]`|\\.)?)*")?)*(?:>|(?=[|()]))|([^    <>|&;(){}'"`\\])(?:(?:(?!\1)[^'`"\\\1|()])*+(?:\\.|'[^']*'|`[^`]`|"(?:[^"\\`]*+(?:`[^`]`|\\.)?)*")?)*(?:\1|(?=[|()]))|(?=\|))|\\.|'[^']*'|`[^`]`|"(?:[^"\\`]*+(?:`[^`]`|\\.)?)*")?)*(?:\)(?:[^  }<>|&;)]|(?:}+(?:[^*?#^~[   <>|&;()${}'"`\\]|(?=[}$'"`\\]))))|[     |(]|$))

\((?=\(((?:[^`'"()$]++|\$\{[^`'"(){}$]+\}|\$(?=[^{`'"()])|`[^`]*+`|\((?1)(?:[)]|(?=['"])))++)(?:[)](?=$|[^)])|["']))|

in python.xml

\w++(?=\s+(?!(?:if|for)\b)[\w'"~]|\s*+(?![.:=\]),]|(?:if|for)\b)((?:(?:ru|u?r|)(?:'(?:[^']++|\\')*+'|"(?:[^"]++|\\")*+")|(?:r?f|fr?)(?:'(?:[^'{]++|\\'|\{\{|\{[^}]++\})*+'|"(?:[^"]++|\\"|\{\{|\{[^}]*+\})*+")|[^#;(){}]|\(\)|\((?1)\)|\{\}|\{(\s*+(?:(?:(?:ru|u?r|)(?:'(?:[^']++|\\')*+'|"(?:[^"]++|\\")*+")|(?:r?f|fr?)(?:'(?:[^'{]++|\\'|\{\{|\{[^}]++\})*+'|"(?:[^"]++|\\"|\{\{|\{[^}]*+\})*+")|[a-zA-Z0-9.]++)\s*+:\s*+(?:(?:ru|u?r|)(?:'(?:[^']++|\\')*+'|"(?:[^"]++|\\")*+")|(?:r?f|fr?)(?:'(?:[^'{]++|\\'|\{\{|\{[^}]++\})*+'|"(?:[^"]++|\\"|\{\{|\{[^}]*+\})*+")|[^#;(){},]|\(\)|\((?1)\)|\{\}|\{(?2)\})++,?)*+)\})+?):)|

in bash.xml

\$\((?=\(((?:[^`'"()$]++|\$\{[^`'"(){}$]+\}|\$(?=[^{`'"()])|`[^`]*+`|\((?1)(?:[)]|(?=['"])))++)(?:[)](?=$|[^)])|["']))|

\((?=\(((?:[^`'"()$]++|\$\{[^`'"(){}$]+\}|\$(?=[^{`'"()])|`[^`]*+`|\((?1)(?:[)]|(?=['"])))++)(?:[)](?=$|[^)])|["']))|

\((?=\(((?:[^`'"()$]++|\$\{[^`'"(){}$]+\}|\$(?=[^{`'"()])|`[^`]*+`|\((?1)(?:[)]|(?=['"])))++)(?:[)](?=$|[^)])|["']))| 

looks like the final pipe is a feature of many of these...

[jgm]

(?1) fails. https://www.regular-expressions.info/subroutine.html (?1) is supposed to match the regex inside the first capturing group.

[jgm]

I've implemented Subroutine and that takes care of everything but this regex from zsh.xml.

 \((?=(?:[^        \\'"|()`ugeP]*+(?:[ug][0123456789]+|[ugeP](?::(?:[^:'`"\\|()]*+(?:\\.|'[^']*'|`[^`]`|"(?:[^"\\`]*+(?:`[^`]`|\\.)?)*")?)*(?::|(?=[|()]))|\[(?:[^]'`"\\|()]*+(?:\\.|'[^']*'|`[^`]`|"(?:[^"\\`]*+(?:`[^`]`|\\.)?)*")?)*(?:\](?=[|()]))|{(?:[^}'`"\\|()]*+(?:\\.|'[^']*'|`[^`]`|"(?:[^"\\`]*+(?:`[^`]`|\\.)?)*")?)*(?:}(?=[|()]))|<(?:[^>'`"\\|()]*+(?:\\.|'[^']*'|`[^`]`|"(?:[^"\\`]*+(?:`[^`]`|\\.)?)*")?)*(?:>|(?=[|()]))|([^     <>|&;(){}'"`\\])(?:(?:(?!\1)[^'`"\\\1|()])*+(?:\\.|'[^']*'|`[^`]`|"(?:[^"\\`]*+(?:`[^`]`|\\.)?)*")?)*(?:\1|(?=[|()]))|(?=\|))|\\.|'[^']*'|`[^`]`|"(?:[^"\\`]*+(?:`[^`]`|\\.)?)*")?)*(?:\)(?:[^  }<>|&;)]|(?:}+(?:[^*?#^~[   <>|&;()${}'"`\\]|(?=[}$'"`\\]))))|[     |(]|$))

[jgm]

Haskell escaped for convenience:

" \\((?=(?:[^        \\\\'\"|()`ugeP]*+(?:[ug][0123456789]+|[ugeP](?::(?:[^:'`\"\\\\|()]*+(?:\\\\.|'[^']*'|`[^`]`|\"(?:[^\"\\\\`]*+(?:`[^`]`|\\\\.)?)*\")?)*(?::|(?=[|()]))|\\[(?:[^]'`\"\\\\|()]*+(?:\\\\.|'[^']*'|`[^`]`|\"(?:[^\"\\\\`]*+(?:`[^`]`|\\\\.)?)*\")?)*(?:\\](?=[|()]))|{(?:[^}'`\"\\\\|()]*+(?:\\\\.|'[^']*'|`[^`]`|\"(?:[^\"\\\\`]*+(?:`[^`]`|\\\\.)?)*\")?)*(?:}(?=[|()]))|<(?:[^>'`\"\\\\|()]*+(?:\\\\.|'[^']*'|`[^`]`|\"(?:[^\"\\\\`]*+(?:`[^`]`|\\\\.)?)*\")?)*(?:>|(?=[|()]))|([^ \t<>|&;(){}'\"`\\\\])(?:(?:(?!\\1)[^'`\"\\\\\\1|()])*+(?:\\\\.|'[^']*'|`[^`]`|\"(?:[^\"\\\\`]*+(?:`[^`]`|\\\\.)?)*\")?)*(?:\\1|(?=[|()]))|(?=\\|))|\\\\.|'[^']*'|`[^`]`|\"(?:[^\"\\\\`]*+(?:`[^`]`|\\\\.)?)*\")?)*(?:\\)(?:[^ \t}<>|&;)]|(?:}+(?:[^*?#^~[ \t<>|&;()${}'\"`\\\\]|(?=[}$'\"`\\\\]))))|[ \t|(]|$))"

[jgm]

The parser says:

Left "Failed reading: parse error at byte position 3"

[jgm]

Isolated the problem to

[^'`"\\\1|()]