pyeve / cerberus

Lightweight, extensible data validation library for Python
http://python-cerberus.org
ISC License
3.16k stars 240 forks source link

Required if another field has value of #194

Closed mczerski closed 7 years ago

mczerski commented 8 years ago

Hi,

this issue is the same as https://github.com/nicolaiarocci/cerberus/issues/121 but the answer is not satisfactory.

I would like to have field "layer" required only if filed "type" has value "folder", so this would be the result: doc = {"type": "folder", "layer": "a"} v.validate(doc, schema) True

doc = {"type": "category"} v.validate(doc, schema) True

doc = {"type": "category", "layer": "a"} print v.validate(doc, schema) False

doc = {"type": "folder"} v.validate(doc, schema) False

from the issue #121 funkyfuture wrote: schema = {'oneof' = [{'product_type': {'regex': '^car$'}, 'color': {'required': True}}, {'product_type': {'regex': '^bike$'}}]}

but this is not even a correct python code. How should i define schema for such case ?

funkyfuture commented 8 years ago

i guess by correcting that syntax error that my example contains.

mczerski commented 8 years ago

sorry, but i have no idea how to do this, i spent already whole day trying to do this. as i understand "oneof" can be used only in field definition, so it is not possible to refer to other fields values - axcept with 'dependecies' (not 'regex'), and unfortunetly it seems that 'dependecies' cannot be used in 'oneof' (it alwaye evaluate to False)

funkyfuture commented 8 years ago

you would have to wrap your document in another one in order to make use of an *of-rule in the scope of a whole socument:

    schema = {'oneof': [...]}
    v.validate({'doc': doc}, {'doc': schema})

i thought there was some remark concerning such usage in the docs, but couldn't spot it ad hox. feel free to contribute an extension.

mczerski commented 8 years ago

ok, thanks. I'll try that

mczerski commented 8 years ago

unfortunetely this doesn't work either, this is my test code:

import cerberus

v = cerberus.Validator()

schema1 = {\
    "type": {"required": True, "allowed": ["folder"]},\
    "layer": {"required": True}\
    }
schema2 = {\
    "type": {"required": True, "allowed": ["category"]},
    }

schema = {"oneof": [schema1, schema2]}

doc = {"type": "folder", "layer": "a"}
v.validate({"doc": doc}, {"doc": schema})
print v.errors

the error is: cerberus.cerberus.SchemaError: unknown rule 'layer' for field 'layer'

seems like validator does not handle this type of schema

funkyfuture commented 8 years ago

which version are you using? schema validation was just changed in the master-branch.

mczerski commented 8 years ago

0.9.2, this is the newest version in pip.

fredrikwidlund commented 8 years ago

I have the same issue, and though I understand the possibility to permute all possible variants and combine with 'oneof' this is not a very clean solution in my use-case, i.e. where the schema is larger, and there are several dependencies of this kind. Also I would like to be able to express my schema as a single external JSON object.

A proposal would be to redefine the 'dependencies' constraint, when the rule is 'required'.

The following..

schema = {
    'name': {'type': 'string', 'required': True},
    'species': {'type': 'string', 'allowed': ['human', 'dog']},
    'owner': {'type': 'string', 'dependencies': {'species': 'dog'}}
}
a = {'species': 'human', 'name': 'Fredrik'}
b = {'species': 'dog', 'name': 'Rover', 'owner':'Fredrik'}
c = {'species': 'dog', 'name': 'Scruffy'}

validator = Validator(schema)
print([validator.validate(x, schema) for x in [a, b, c]])

Results in...

[True, True, True]

Which is fine, however I would like to leave Scruffy out in the cold, and require that all dogs have an owner (i.e. achieve [True, True, False]). I would like to be able to do the following:

schema = {
    'name': {'type': 'string', 'required': True},
    'species': {'type': 'string', 'allowed': ['human', 'dog']},
    'owner': {'type': 'string', 'dependencies': {'species': 'dog'}, 'required': True}
}

However this would falsify everything but dogs, resulting in [False, True, False].

So my suggestion is that the semantics for a required rule containing dependencies means

I.e. a dog has to have an owner.

This doesn't take away anything from the current semantics since a rule, both containing dependencies and being required, currently isn't very meaningful.

I.e. currently...

schema = {
    'a':{'type':'string'},
    'b':{'type':'string', 'dependencies':['a'], 'required': True}
}

...is the same as...

schema = {
    'a':{'type':'string', 'required': True},
    'b':{'type':'string', 'required': True}
}

unless I'm misunderstanding something.

fredrikwidlund commented 8 years ago

Looking at the code, this is actually how I read the description, but either I'm missing something or I'm not able to make it work?

def __validate_required_fields(self, document):
""" Validates that required fields are not missing. If dependencies
    are precised then validate 'required' only if all dependencies
    are validated.

    :param document: The document being validated.
"""
funkyfuture commented 8 years ago

is this the same need as expressed in #182?

fredrikwidlund commented 8 years ago

182 is more open. This is a specific suggestion. Though personally I'm going for http://json-schema.org/latest/json-schema-core.html instead. Using combinations of oneof/anyof etc is possible in some cases, but not always.

fredrikwidlund commented 8 years ago

This could be seen as a bug as well, since the comment after __validate_required_fields implies this behaviour, even though it seems not to be implemented.

stijnvanhoey commented 8 years ago

With an install of the git-master version (cerberus.__version__ >>> '0.10'), the testcase of @mczerski (I only adapted the validation by first creating the instance):

import cerberus

v = cerberus.Validator()
schema1 = {"type": {"required": True, "allowed": ["folder"]}, "layer": {"required": True}}
schema2 = {"type": {"required": True, "allowed": ["category"]}}

schema = {"oneof": [schema1, schema2]}
doc = {"type": "folder", "layer": "a"}
v = Validator(schema)
v.validate(doc)
print v.errors

is now resulting in the following error message:

...   raise SchemaError(self.schema_validator.errors)
SchemaError: {'oneof': 'must be of dict type'}

I don't really get the idea of the syntax: v.validate({'doc': doc}, {'doc': schema})?

funkyfuture commented 8 years ago

I don't really get the idea of the syntax: v.validate({'doc': doc}, {'doc': schema})?

an of-rule can only be defined on a field-level, but not the document-level. the initial question asked for variants that stretch across multiple top-level fields. thus you need to 'fake' a field that covers the whole document and use the schema-rule inside the of-definitions to address the no-more-top-level fields.

and it seems to me that is why your example doesn't work.

omorillo commented 7 years ago

I am also in need of this feature. What's the current state of this issue @mczerski @funkyfuture ?

linclelinkpart5 commented 7 years ago

I seem to be running into this issue too. I want a field to be present and required iff a specific sibling field has a particular value:

from cerberus import Validator
from pprint import pprint

schema = {
    'procedure': {
        'type': 'list',
        'required': True,
        'schema': {
            'type': 'dict',
            'schema': {
                'sigil': {
                    'type': 'string',
                    'minlength': 4,
                    'maxlength': 4,
                    'allowed': ['INGR', 'UNOP', 'BNOP', 'DIVD', 'RESV', 'DISC', 'PSEU', 'LKBK'],
                    'required': True,
                },
                'name': {
                    'type': 'string',
                    'minlength': 1,
                    'required': True,
                    'dependencies': {
                        'sigil': ['INGR', 'UNOP', 'BNOP', 'PSEU'],
                    },
                },
                'modifiers': {
                    'type': 'list',
                    'default': [],
                    'schema': {
                        'type': 'string',
                        'minlength': 1,
                    },
                    'dependencies': {
                        'sigil': ['INGR', 'UNOP', 'BNOP'],
                    },
                },
                'index': {
                    'type': 'integer',
                    'min': 0,
                    'required': True,
                    'dependencies': {
                        'sigil': 'LKBK',
                    },
                },
            },
        },
    },
}

v = Validator(schema)

documents = [
    { 'procedure': [ { 'sigil': 'INGR', 'name': 'onions', }, { 'sigil': 'UNOP', 'name': 'chop', }, ], },
    { 'procedure': [ { 'sigil': 'INGR', 'name': 'onions', }, { 'sigil': 'INGR', 'name': 'tomatoes', }, ], },
    { 'procedure': [ { 'sigil': 'UNOP', 'name': 'chop', }, ], },
    { 'procedure': [ { 'sigil': 'BAD!', 'name': 'error', }, ], }, # Testing a fail case
]

for document in documents:
    result = v.validate(document)

    if result:
        print("Document validated successfully!")
    else:
        pprint(v.errors)

This outputs:

{'procedure': [{0: [{'index': ['required field']}],
                1: [{'index': ['required field']}]}]}
{'procedure': [{0: [{'index': ['required field']}],
                1: [{'index': ['required field']}]}]}
{'procedure': [{0: [{'index': ['required field']}]}]}
{'procedure': [{0: [{'index': ['required field'],
                     'modifiers': ["depends on these values: {'sigil': "
                                   "['INGR', 'UNOP', 'BNOP']}"],
                     'name': ["depends on these values: {'sigil': ['INGR', "
                              "'UNOP', 'BNOP', 'PSEU']}"],
                     'sigil': ['unallowed value BAD!']}]}]}

I would have expected the first three cases to validate, especially since the docs note that required isn't checked if dependencies aren't met. Am I understanding correctly?

funkyfuture commented 7 years ago

considering that time has passed, things have changed and different users asked different questions, i propose to close this issue.

anyone with usage question may ask on SO, anyone encountering a specific bug may open an issue here.

jeking3 commented 5 years ago

Too bad, I could have used a mechanism like this.

nicoCalvo commented 5 years ago

I went through the whole thread, the documentation and even SO. It's such a shame this is not being treated as supposed, there's no sense in using a validator that cannot make a difference for fields depending on others.

I did tried the suggestion and I could finally make it work, but at the end of the "oneof" hack, I get this useless error message:

{'account': ['none or more than one rule validate', {'oneof definition 0': [{'skip_first': ['unknown field']}], 'oneof definition 1': [{'due_date_rule': ['required field']}]}]}

   credit = {'active': {'type': 'boolean', 'required': False},
              'credit_limit': {'type': 'integer', 'required': True},
              'due_date_rule': {'type': 'string', 'validator': validate_rule, 'required': True},
              'skip_first': {'type': 'boolean', 'required': True},
              }
    cash = {'active': {'type': 'boolean', 'required': False},
            'credit_limit': {'type': 'integer', 'required': True},
            }

    NEW_SCHEMA = {'customer': {'required': True, 'coerce': User.get_instance},
                  'account': {'oneof_schema': [cash, credit], 'type': 'dict'},
                  'credit_type': {'required': True, 'coerce': coerce_account}
                  }

The scenario being tested is "credit" with only a missing field, the "due_date_rule". That message has no sense.

jeking3 commented 5 years ago

I ended up wrapping Cerberus with my own post-cerberus validator that could implement custom complex conditional logic, using code instead of YAML. It works well enough and lets me do things even simple conditional logic would not be able to do. I could perhaps see something like cerberus implementing either a yaql or an objectpath conditional clause, that would give it more power for complex logic.

alfredo-g-zapiola commented 5 years ago

This worked for me https://stackoverflow.com/questions/48789686/how-to-make-cerberus-required-rule-depends-on-condition

Quronox commented 4 years ago

For Everyone in the Future, this is the Answer: https://stackoverflow.com/questions/61954183/is-it-possible-to-set-conditional-validation-in-cerberus-python