Support ECMAScript unicode-mode RegExp usage for 'pattern' and 'patternProperties'

[djgoku]

Instead of duplicating everything here is my problem.

If you want me to copy anything for this issue I can.

https://github.com/awslabs/amazon-ecs-intellisense-schema/pull/8

I am looking to see how I can support Python Unicode regex so I can use this json schema.

[sirosen]

If I have understood the issue correctly, this is a matter of a schema using an ECMA regex syntax which python (the language) does not support.

However, luckily, we're using regress here to provide ECMA-compatible regex support for format. Which means that I need to know what type of match is failing, and then with through instrumenting the right support.

[djgoku]

Thanks for looking into this. I am opening to helping if I can. I'll try to check out the errors again tomorrow.

[sirosen]

To put all the information for this in one place (with some fun emojis):

• :mag: The issue in question is that pattern usage needs to support unicode-mode regexps, which can contain special escape patterns
  • :book: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp/unicode#description
• :crossed_fingers: Will need to assess whether or not regress already implements this
• :hammer_and_wrench: There is a well-defined path for plugging in an alternative implementation of pattern and patternProperties
  • :nerd_face: This looks interesting and doable, but will require some careful reading through jsonschema source
• :question: I still need some nice MRE for this issue so that I can test the results
  • :partying_face: that's a great place to help out if anyone has time -- create a small example of a schema with a regex which requires unicode-mode, and positive and negative cases for that schema

[Rojax]

I'm having a similar issue described here: https://github.com/usnistgov/metaschema/issues/770.

In short: Having this

"pattern": "^(\\p{L}|_)(\\p{L}|\\p{N}|[.\\-_])*$" 

results in this

Error: schemafile was not valid: '^(\\p{L}|_)(\\p{L}|\\p{N}|[.\\-_])*$' is not a 'regex'
Failed validating 'format' in metaschema['properties']['definitions']['additionalProperties']['properties']['pattern']:
    {'type': 'string', 'format': 'regex'}
On schema['definitions']['TokenDatatype']['pattern']:
    '^(\\p{L}|_)(\\p{L}|\\p{N}|[.\\-_])*$'
SchemaError: '^(\\p{L}|_)(\\p{L}|\\p{N}|[.\\-_])*$' is not a 'regex'
Failed validating 'format' in metaschema['properties']['definitions']['additionalProperties']['properties']['pattern']:
    {'type': 'string', 'format': 'regex'}
On schema['definitions']['TokenDatatype']['pattern']:
    '^(\\p{L}|_)(\\p{L}|\\p{N}|[.\\-_])*$'

[sirosen]

I took some time today to do the internal restructuring I've been meaning to do, in order to make this possible. It's most of the way there, but I've hit a bit of a strange case with custom validators which I need to sort out. And I still need to put together a good test case to verify my new work.

I think this will not work with arbitrary custom validators, at least for the initial version. In order to attach the alternate pattern validation to a validator class, I'm using the extend API. In theory, a custom validator class could have changes which that API will not preserve. I need to work out how to document this, since it's subtle.

And the other notable thing here is that this is a change to pattern but not patternProperties, at least at the moment. The two are different and each requires it's own implementation, though they can share some bits.