Metadata/tag support for fields in the schema

[draggeta]

I'm loving the performance of validx, but there is one thing that stops me from using it. For a project we need to be able to specify metadata (form doesn't matter, could be tags as well) for the schema on each level. We need to output this metadata if a field doesn't match.

An example would be that a value must be an Int between 10 and 20 and if it isn't, output the error as well as the metadata detailing it as non-critical.

I haven't been able to find anything like that in the docs, but I may be wrong.

[draggeta]

Managed to get something working by creating a wrapper class:

class ValidxWrap(validx.cy.Validator):
    """
    Wraps classes in ValidX 
    """
    step: validx.cy.Validator
    metadata: t.Optional[t.Dict]

    def __init__(
        self,
        step: validx.cy.Validator,
        metadata: t.Optional[t.Dict] = dict(),
    ) -> None:
        self.step = step
        self.metadata = metadata

    def __call__(self, value, _):
        try:
            return self.step.__call__(value)

        except Exception as e:
            e.metadata = self.metadata
            raise e

and then using it like this:

Dict({
    "sequence-id": ValidxWrap(Int(options=[0]), metadata={"critical": True}),
    "description": ValidxWrap(Str(options=['somestring']), metadata={"critical": False}),
})

[kr41]

Do you have any idea of how it should work on nested validators?

schema = Dict(
    {
        "x": Int(metadata={"somevalue": 1}),
    },
    metadata={"somevalue": 2}
)
try:
    schema({"x": None})
except SchemaError as error:
    print(error[0].metadata)

Should it be {'somevalue': 1} or {'somevalue': 2} or something else?

[draggeta]

In this case I'd like for the Int() validator on x to return that metadata. I'm not sure what to do if the dict and schema also has a key y and x is missing altogether.

schema = Dict(
    {
        "x": Int(metadata={"somevalue": 1}),
        "y": Str(metadata={"somevalue": 3}),
    },
    metadata={"somevalue": 2}
)
try:
    schema({"y": "text"})
except SchemaError as error:
    print(error[0].metadata)

[kr41]

I was thinking about the problem yesterday, and I don't see any clear and general solution with metadata as dict. One user would expect the metadata of nested validators should be merged with leaf priority, i.e. {"somevalue: 1} in the example above, another one would expect it should be merged with root priority — {"somevalue": 2}. Moreover, I have no idea how to describe the feature in the documentation, and if it's hard to document, then it has bad design.

However, the idea with tags looks pretty obvious. We just treat them as sets of strings and merge all nested tags together.

schema = Dict(
    {
        "x": Int(tags={"x-tag"}),
        "y": Int(tags={"y-tag"}),
    },
    tags={"schema-tag"}
)
try:
    schema({"x": "1", "y": "2"})
except SchemaError as error:
    error.sort()
    print(error[0]) # <x: InvalidTypeError(..., tags={"x-tag", "schema-tag"})>
    print(error[1]) # <y: InvalidTypeError(..., tags={"y-tag", "schema-tag"})>

[draggeta]

Ah, that is a good point. For us, we're mostly interested in the metadata for the place the error occurs.

schema = Dict(
    {
        "x": Int(metadata={"somevalue": 1}),
        "y": Str(metadata={"somevalue": 3}),
    },
    metadata={"somevalue": 2}
)
try:
    schema({"y": "text"})
except SchemaError as error:
    print(error)

With the following output:

<SchemaError(errors=[
    <y: InvalidTypeError(expected=<class 'str'>, actual=<class 'int'>)>,    <-- contains metadata {"somevalue": 3}
    <x: MissingKeyError()>                                                  <-- contains metadata {"somevalue": 2} # I believe the dict is throwing this error?
])>

There would be no merging. Each error would have it's own metadata depending on which part fails. If x isn't set, it may be {"critical": True} , but if it is set, but to the wrong value, it could be {"critical": False} for example.

However, I do concede that our use case may be very specific. I do like the tag implementation you specified as well. It might be handy in that case to have a list of tuples of tags. That way you could find out which tags are for which nesting level: for example: [("critical", "impacting"),("non-critical", "security")]

[Yoyasp]

Hi there. I am a co-worker of @draggeta , thought i would pitch in aswell. As he said we would like for the error itself to contain the metadata. This will also enable custom error messages to be used without implementing a new type validator.

For example:

schema = Dict(
    {
        "username": Str(pattern='user-.*', metadata={"custom-error": "Company policy states all usernames must start with user-"}),
    },
)
try:
    schema({"username": "test"})
except SchemaError as error:
    if error.metadata['custom-error']:
        print(error.metadata['custom-error'])
    else:
        print(error)