Open kjosib opened 5 years ago
If AST nodes are defined for bits of character-class structure (rather than "compiled" classes), then visitors may be defined to interpret those nodes in an any manner. In particular, curly-brace named references can be the province of semantic interpretation rather than a hack in the meta-lexer. This opens up more possibilities for Unicode-oriented scanning.
Regular expressions now get completely turned to AST nodes rather than building character classes during the parse, so all reference names get delayed binding, so just don't make cycles, eh? (Although I suppose technically tail-recursion could be made to work, I don't think it's worth the effort.) This means the metascanner no longer uses the monkey-patch, and miniscan no longer supports it. A case-sense switch may not be far off.
Related: a recent commit 57b0b71e6a188570d1395eccfc958c5ac47290f5 has switched from using hand-coded named-tuple based AST nodes to now using the arborist.trees
subsystem, which didn't change any user-visible functionality but might bring this issue slightly closer to completion.
scanning.miniscan.Definition.scan(...)
provides for a dirty trick: monkey-patching an extra attribute into an object defined elsewhere.scanning.miniscan.__BEGIN__._bracket_reference
relies on that monkey-patch. That results in having to define subexpressions before they occur (which may sound like a fine thing, but isn't ideal:).The metascanner problem be fixed by making regex AST nodes expect an environment passed into their translate-to-NFA method, which also facilitates nice things like case-insensitive sub-patterns.
I'm not sure if the monkey-patch scanner.env attribute gets used in the wild. Does it?