NDB should consider providing a data validation hook

[GoogleCodeExporter]

I'm currently using _pre_put_hook to make sure my keys have the right id and 
ancestry structure. This causes an exception when I try to insert entities into 
the datastore, rather than where I actually make the mistake. It would be 
better to have the model class validate its invariants where I change is, so I 
get an exception with a precise stack trace.

On the other hand, checking at every property update would likely cause false 
positives. (For example, when changing a date from 2012-03-30 to 2012-02-25, 
validating on every update could flag an error at an intermediate state of 
2012-02-30. On the other hand, using populate to set the new values would skip 
the intermediate invalid state.) Checking after a get() could also make it 
impossible to retrieve invalid values that were written to the datastore by a 
previous version of a model. So this kind of hook would have to be handled 
carefully.

This is currently a theoretical feature request, since I haven't actually had 
problems yet from the mis-located exception.

Original issue reported on code.google.com by jyass...@gmail.com on 15 Apr 2012 at 10:54

[GoogleCodeExporter]

Somehow I missed this in the tracker. Could you elaborate a bit more on a 
possible API for this feature that would satisfy your requirements? Could you 
perhaps implement a version of it for a specific model class in user code to 
show the pattern?

Original comment by guido@google.com on 27 Sep 2012 at 5:50

[GoogleCodeExporter]

I also starred this issue, and my issue was actually very similar (altho less 
specific and more general).

I was looking for a data validation hook that allows me to "sanitize" data 
going into a data model (e.g. ensure that a field contains a valid phone 
number, but then morph that data down into an Integer for example).

Pseudo Code:

m = Model()
m.phonenumber = "(480) 555 1212"
>> Validate: Has 10 digits? Yes, lets strip it down to an "int" and re-write 
the property
m.put()

m = Model()
print m.phonenumber
>> 4805551212

Then, I could write a Jinja filter (or similar) to display the value how my 
template chooses.

Original comment by ric...@onixnet.com on 27 Sep 2012 at 6:06

[GoogleCodeExporter]

Ah, but if the validation is at the property level you can do this very easily 
by defining your own subclass of StringProperty() of IntegerProperty() and 
overriding the _validate() method. Just make the latter raise an exception if 
the value is incorrect and return the normalized value otherwise. See 
https://developers.google.com/appengine/docs/python/ndb/subclassprop

Original comment by guido@google.com on 27 Sep 2012 at 6:22

[GoogleCodeExporter]

Ahh, Makes sense.  This is a better fit for my specific use case (e.g. 
PhoneNumberProperty)

Not too sure about OP's use case :D

Original comment by ric...@onixnet.com on 27 Sep 2012 at 6:30

[GoogleCodeExporter]

Ironically, the old db package has a PhoneNumberProperty but it doesn't have 
any validation except insisting on a non-empty value, which is exactly the 
opposite from what you want. Also note that even if you do implement a 
validating PhoneNumberProperty, you're still going to want to implement a 
separate validation routine in your UI so users get reasonable error messages 
(preferably in JavaScript so they won't need a server roundtrip). Undoubtedly 
web frameworks have form features to support this. But it's still necessary to 
double-check the validation on the server-side (although a good web framework 
would use the same specification to generate both the JavaScript and the 
server-side validation code).

I think Jeffrey wants to check constraints across properties, and things that 
aren't properties (e.g. the key id and the ancestor).

I think the key id and ancestor could probably be validated by overriding the 
_key and key fields: these both point to a ModelKey object; that class 
subclasses Property and you can probably just override the _validate() method 
in the subclass. But that still leaves validation involving multiple properties 
as a use case. (Also subclassing ModelKey is undocumented. :-)

Original comment by guido@google.com on 27 Sep 2012 at 6:40

[GoogleCodeExporter]

I'll "bump" this and say that having an ability to validate the model would be 
very helpful for different models. For example, modeling a meeting with start 
time and end time, would be helpful to validate that end time is not before 
start time.

Having a per property validator would not be very useful because that relies on 
assignment ordering. 
My current solution is to override the put method, but as mentioned this is 
pretty bad for error handling.

Original comment by AceP...@gmail.com on 18 Apr 2014 at 1:16

[GoogleCodeExporter]

I'd alos like a validation hook at the model level.
This is my workaround: http://stackoverflow.com/a/25786896/2003429

Original comment by a...@schenkman.info on 11 Sep 2014 at 12:00