The Profile code uses regex's [[:upper:]] and [[:lower:]] to test characters, but not all Unicode alphabetic characters map to either. Those characters are therefore passed along as-is. The problem is, the pattern field itself is a STRING, so HPCC performs a coercion of those as-is characters and you wind up with something different (valid coercion, I-Don't-Know characters, etc).
One solution is to convert the pattern field to UTF8. Another is to find a way to map those types of Unicode characters better.
dcamper
commented
11 months ago
The Profile code uses regex's [[:upper:]] and [[:lower:]] to test characters, but not all Unicode alphabetic characters map to either. Those characters are therefore passed along as-is. The problem is, the pattern field itself is a STRING, so HPCC performs a coercion of those as-is characters and you wind up with something different (valid coercion, I-Don't-Know characters, etc).
One solution is to convert the pattern field to UTF8. Another is to find a way to map those types of Unicode characters better.