advice on validating attributes before put / batch write?

[wmclendon]

hello,

apologies if this is a bit of an elementary question. i would call myself relative novice to python, and programming in general, but trying to learn and understand more.

i've been teaching myself some Amazon skills and coding alongside, and i'm trying a toy project that involves API Gateway, Lambda, DynamoDB, and onsite client scripts that will post data to the API Gateway, which then calls the Lambda, which parses it out into my Model and then writes to DynamoDB

My question is around validating that the input I received and assigned into the instance's attributes are valid (so far as the model believes) before attempting to write to dynamo. PynamoDB seems to do this validation when the write is attempted -- if I've defined an attribute as UnicodeAttribute, and it gets set to an integer, it fails as expected when trying to serialize. Is there a way for me to force a serialization effort of all the attributes that I can put in a try block or something as a "check" before attempting to write?

In my use case multiple items would be POSTed to the API at once as a list of dictionaries, and i'd like to batch write them (I assume that is best vs iterating and writing one by one in a loop?), and if one fails for now its acceptable to fail out the whole batch, but i'd like to be able to log or return back which item and / or item attribute caused the failure.

Or is there a better way I should handle this ahead of trying to create the object and write to DynamoDB? My understanding is there are ways to define schemas or classes that can do that (Marshmallow is one I think?), but I feel like i've already done that effort by defining the DynamoDB model itself. If however that is how it should be done, i'll proceed down that path (pointers / helpful hints appreciated!).

Thanks!

[ikonst]

The kind of validation that PynamoDB attributes perform is light relative to the kind of validation you can (and do) perform with e.g. marshmallow's validators. So the range of possible errors is not that big -- basically type errors and null errors.

For type errors like assigning an int to a UnicodeAttribute, type checkers like mypy can help. Of course you can also derive MyUnicodeAttribute with a __set__ override that'll perform a type assertion or any other validation. This kind of validation would apply at assignment time rather than at serialization time, i.e. closer to where you actually made the error.

[andre-at-clubien]

The way I do this is by using marshmallow or, if you are using flask, flask_marshmallow (which is basically marshmallow under the hood, but tweaked to interact more 'easily' with Flask). It might look a bit scary at first, but just try it... it will make sense.