Marshalling vs Parsing

[Elfayer]

I have some comprehension issue related to marshalling and parsing.

Marshalling

Let's start with the marshalling. It is defined in the doc that it should be used "to control what data you actually render in your response or expect as in input payload". So it is for input and output then? Why is the section called "Response Marshalling" if it also takes care of the input?

So on my end points, I do things as follow:

@namespace.expect(user)
@namespace.marshal_with(user_full)
def put(self):

I have a different schema expected for input and marshalled for output.

Parsing

Now here comes the thing. I defined what I want on entry, and output. What do I need Parsing for?

If my user schema looks like:

user = namespace.model('User', {
    'id': fields.Integer(required=True),
    'first_name': fields.String(required=True),
    'last_name': fields.String(required=True),
    'email': fields.String(required=True),
    'age': fields.Integer(required=True)
})

Why do I need to define the parsing as follow, while the model is already defined?

parser = reqparse.RequestParser()
parser.add_argument('id', required=True)
parser.add_argument('first_name', required=True)
parser.add_argument('last_name', required=True)
parser.add_argument('email', required=True)
parser.add_argument('age', required=True, type=int)
args = parser.parse_args()

Couple of things seems weird to me here:

• It looks extremely verbose to me. What is the point to expect a model if you have to parse it again?
• Why do I have to redefine the type, the required, everything actually?
• Why not simply: args = parser.get_expected_model()? (just an example: one line, instead of rewriting the whole model)

There must be something I don't understand, could someone clarify this for me?

[tonoli]

I have the same issue.

It's not very clear how to use these two tools. So for the moment, I did exactly the same thing than your examples.

[alessbelli]

+1, is there a shorthand for all this?

[avilaton]

You don't need to write all of this:

parser = reqparse.RequestParser()
parser.add_argument('id', required=True)
parser.add_argument('first_name', required=True)
parser.add_argument('last_name', required=True)
parser.add_argument('email', required=True)
parser.add_argument('age', required=True, type=int)
args = parser.parse_args()

It is enough with using the decorator @namespace.expect(user) to get input validation. Please have a look at the example in http://flask-restplus.readthedocs.io/en/stable/example.html?highlight=expect

[MarlieChiller]

Having this same issue. In your example at http://flask-restplus.readthedocs.io/en/stable/example.html?highlight=expect endpoints have both an expect and a marshall_with. Dont they effectively do the same thing?

[marius-nicolae]

Looking in the source code, it looks like the expect decorator is doing only Draft4 json schema validation as described at https://python-jsonschema.readthedocs.io/en/stable/validate/#versioned-validators, triggered by the flask_restplus.model.validate function. I can't see how to add custom validators.

The _marshalwith decorator seems to be the way to go, but there is a catch. The methods have to return a "marshalable" object. I've spent a lot of debugging time because my post method was returnig "None, 201". I missed the

A decorator that apply marshalling to the return values of your methods.

line from https://flask-restplus.readthedocs.io/en/stable/api.html#flask_restplus.marshal_with documentation page. Another big drawback is that the validation of the input data ( api.payload) takes place on method return, after the invalid data was stored in database (as it is my case).

The cleanest method would be to allow _marshalwith decorator work with objects like marshmallow schemas for get method. The validation will automatically take place when issueng schema load(s).

Until then, do you see any workarrounds for performing both input and output data validation but to use a single schema/model definition?

[marius-nicolae]

With the hope it will be useful to someone I think I've found a workaround. The objectives were:

• describe the input/output data model/schema only once (single point of truth)
• keep the swagger documentation functionality (i.e. employs _api.marshalwith decorator with a flask_restplus model on resources get methods and api.expect on post/put methods)
• on put/post:
  • validate the data before storing it to the database (before method return anyway)
  • allow None return while still validating the input data
• use the more powerful marshmallow fields with their validators
• transport flask_restplus fields specific attributes (like title, description, example etc.) through marshmallow fields

The workaround consists of creating flask_restplus api model out of a marshmallow schema using a helper function defined as:

from marshmallow import Schema, fields
from marshmallow.exceptions import ValidationError
from flask_restplus.fields import Raw as RawField, MarshallingError

def ModelFromSchema(Schema):
    class FlaskRestPlusField(RawField):
        __schema_type__ = 'string'

        def __init__(self, mmField: fields.Field):
            super().__init__(attribute=mmField.attribute,
                             required=mmField.required, readonly=mmField.load_only, **mmField.metadata)
            self.m_mmField = mmField

        def format(self, value):
            try:
                self.m_mmField._validate(value) #should throw in case of error
            except ValidationError as e:
                raise MarshallingError(e)
            return value
    res={}
    for fieldName, field in schema._declared_fields.items():
        res[fieldName] = FlaskRestPlusField(field)
    return res

An usage example might be:

class MarshmallowSchema(Schema):
    id = fields.Integer(description='The unique identifier')
    name = fields.Str(required=True, description='Example name')
    email = fields.Email(required=True, description='Example email')

flask_restplusModel = api.model("Example model", ModelFromSchema(MarshmallowSchema))

The flaskrestplusModel can now be used with @api.marshal_listwith decorator on a reosurce get method while the MarshmallowSchema schema object can be used to load (deserialize) and validate input data (api.payload) and raise a ValidationError exception in case the validation fails.