Regex Unicode properties

[tjpalmer]

It would be nice to support Unicode properties for advanced natural language text parsing, but we have some potential problems:

• Python has no built-in support for these, though we seem to be in agreement that requiring 3rd-party regex isn't a bad option.
• .NET has no built-in support for code points past 16 bits, so their Unicode properties aren't reliable, and we haven't found a good 3rd party yet. Giant regex explosion is maybe an option or else pcre2 or our own engine.
• We can't fully promise compatible Unicode tables even across backends with good support.
• One related issue is that these things also make grapheme cluster matching hard.

Some solutions:

• Live with certain amounts of incompatibility across backends.
• Implement a full regex engine in Temper once it's strong enough for that. This full regex engine might or might not be distributed with Temper itself. @ShawSumma has written an initial engine.

[ShawSumma]

Funny enough I spent the weekend exploring regex engines.

I've managed to get all the suggested ASCII categories working. [[:digit:]ab] is a class that matches a base 12 digit for instance.

It can be found on the (currently private) temper-regex-engine branch named ext.

[tjpalmer]

Sounds good. I'm especially concerned with things I think people are likely to need. And some unicode categories fit into that if people want to deal with arbitrary human language.