Allow for missing values

[scott2b]

It is not clear why colander strictly enforces the presence of non-required values. There should at least be an 'optional' parameter which would enable validation of schemata with non-required nodes without forcing undesired default or sentinel values.

[mcdonc]

Can you provide a bit of sample code that doesn't behave as you'd like/expect? Does your schema have "missing=" values?

[mcdonc]

No response, closing.

[scott2b]

Chris, sorry I did not get back to you on this. It was a weird week and my inbox is kind of piled up. The issue was that missing= is really treated as a default=, meaning that if the value is not provided, then a default is provided. My feeling is that it would be really useful to actually allow for missing values which do not have a default. The primary use case I have is in validating schema for NoSQL or similar solutions where you don't want empty values cluttering up the data. To do this, I am defaulting to None, and then just popping the keys with None values before pushing into a MongoDB.

This is inconvenient but it works. I think this really goes in the category of design choice more than bug.

[mcdonc]

Ah, right! I'l reopen this. It's a reasonable feature request (albeit one we might not be able to get around to in any predictable amount of time). I might suggest something like "missing=colander.drop" for child nodes of sequences and mappings. If subnode of a sequence or mapping deserializes to "colander.drop", the container implementation would just omit it from the deserialization value.

[scott2b]

That seems like a good approach.

[kusut]

Request extended.

Say you've implemented colander.drop and people are able to set it on their nodes. Would it be convenient to have a schema level setting to make all nodes dropable?

Here be example:

class Foo(Schema):
    bar = String
    baz = String

The idea is to make this schema works normally as it currently does, and provides a switch to make all nodes droppable when missing.

Maybe give deserialize in _SchemaNode another kwarg:

class _SchemaNode(object):
    deserialize(self, cstruct=null, drop=False)

So it will be like this:

>>> f = Foo()
>>> f.deserialize({'bar': 'abc'}, True) # all missings, in this case baz, will be dropped
{'bar': u'abc'}

Note: maybe raising invalid if all nodes are missing? Another kwarg?

This way, we can use the same schema on both creating and updating an object.

Is this a good idea? Thoughts?

[AndreLouisCaron]

+1 for the deserialize(..., drop=True) suggestion. To me, this is not a property of the schema, but rather a filter on the result of deserialization.

[mcdonc]

The missing=colander.drop feature was implemented. I'm going to close this issue as a result. If the suggestions above need further consideration, please open new issues with them attached.