Document integration with other libraries

[sloria]

e.g., How does marsmallow-jsonapi integrate with:

• flask-marshmallow
• marshmallow-sqlalchemy

[jfinkels]

This is relevant to me. I'm the maintainer of Flask-Restless and I'd like to put together an example server configuration that uses marshmallow-jsonapi with marshmallow-sqlalchemy.

[Dr-ZeeD]

How would one use marshmallow-jsonapi together with flask-marshmallow and marshmallow-sqlalchemy? I seem to always get this error:

AttributeError: 'SchemaOpts' object has no attribute 'inflect'

[jjb122]

I've been struggling for the last few days now to get these three libraries working together, and continually get issues like the one @ZeeD26 mentioned above, and some others regarding Schemas - Seems like the ModelSchema in flask-marshmallow and Schema class in Marshmallow-JsonAPI don't quite integrate together when I have a subclass that inherits from both of these classes. Anyway, any sort of documentation/examples/advice would definitely be appreciated.

[rgant]

I've documented my implementation of integrating marshmallow-jsonapi and marshmallow-sqlalchemy: https://github.com/rgant/saas-api-boilerplate/blob/master/ourmarshmallow/README.md

I have decent test coverage and am not seeing any issues like @ZeeD26 or @jjb122 were. I've used a previous version of this for more than a year with great success.

The problem is that it required a number of assumptions to produce automatic settings. If there is a desire I can create a pull request that documents a more direct setup for others to use. I just worry that this won't be broadly applicable unless you are doing JSONAPI exactly the same way as I am (all relationships implemented). Which is why I expect this to be documentation, not an extension or plugin for this module.

Feedback would be great.

[rgant]

@sloria I'm a bit hesitant to just add my one-true-way of integrating marshmallow-jsonapi and marshmallow-sqlalchemy to the documentation here without oversight and review. Interest was expressed in my repo, and I'm about to start doing JSONAPI/SQLAlchemy python API development again soon. So I think in a month or so I would be in a good position to polish up my methods for general use.

[jmehnle]

@rgant, do you have an update to share?

[rgant]

There didn't seem to be any interest and I haven't made this a priority @jmehnle. I'd need to go through 2 years of Marshmallow updates in three projects to get this going again. It might make the most sense to just start over.

[antgel]

@sloria As the OP, what are your current views on this? Following recent discussions on Slack, I'm keen to explore marshmallow-jsonapi, but I'm already pretty invested in marshmallow-sqlalchemy, and I'm using the stack for POC code, so no urgency, just interesting to know what integration might look like.

[sloria]

Unfortunately, I don't really time to invest in this at the moment (I'm not using this project). I'd definitely merge documentation if someone is up to it, but won't be working on it myself in the near future.

[sloria]

That said, I don't think there's any special magic needed to integrate these. Subclassing from e.g. ModelSchema and Schema should work fine. You'll just need to make sure to inherit the class Metas.

from marshmallow_jsonapi import Schema, fields

class ArtistSchema(ma.ModelSchema, Schema):
    class Meta(ma.ModelSchema.Meta, Schema.Meta):
        model = models.Artist
        type_ = 'artists'

    albums = fields.Relationship(...)

If you don't want marshmallow-sqlalchemy to autogenerate fields for relationships, you can use TableSchema.

from marshmallow_jsonapi import Schema, fields

class ArtistSchema(ma.TableSchema, Schema):
    class Meta(ma.TableSchema.Meta, Schema.Meta):
        table = models.Artist.__table__
        type_ = 'artists'

    albums = fields.Relationship(...)

Note: TableSchema won't deserialize to model instances, but that's easy enough to add with a post_load method or in app code. We might add an option to disable relationship introspection for ModelSchema (https://github.com/marshmallow-code/marshmallow-sqlalchemy/issues/98).

[antgel]

@sloria Thanks for that, however in my trivial example, I got the same error as @ZeeD26 .

AttributeError: 'SchemaOpts' object has no attribute 'inflect'

Did you run the example code, or was it just supposed to work? :) If the former, I'll post a longer example of my code, but it seems that this is a pretty common error when trying to inherit from both marshmallow-jsonapi and marshmallow-sqlalchemy.

Note that if I swap the superclasses around i.e. class ArtistSchema(Schema, ma.TableSchema):, I get 'SchemaOpts' object has no attribute 'model_converter'. I suspect that someone who understands inheritance better than me can fix this quite easily.

Edit: Here's some stack trace which the marshmallow gurus can parse:

  File "/home/antony/.virtualenvs/lib/python3.6/site-packages/marshmallow/schema.py", line 938, in _init_fields
    self._bind_field(field_name, field_obj)
  File "/home/antony/.virtualenvs/lib/python3.6/site-packages/marshmallow/schema.py", line 987, in _bind_field
    self.on_bind_field(field_name, field_obj)
  File "/home/antony/.virtualenvs/lib/python3.6/site-packages/marshmallow_jsonapi/schema.py", line 224, in on_bind_field
    field_obj.data_key = self.inflect(field_name)
  File "/home/antony/.virtualenvs/lib/python3.6/site-packages/marshmallow_jsonapi/schema.py", line 277, in inflect
    return self.opts.inflect(text) if self.opts.inflect else text
AttributeError: 'SchemaOpts' object has no attribute 'inflect'

[multimeric]

I think to fix the latter issue (missing inflect or model_converter) you'll also need the SchemaOpts class to inherit from both parents. I haven't tested it, but it should be something like:

from marshmallow_jsonapi import schema as ja_schema
from marshmallow_sqlalchemy import schema as sql_schema

# Both child classes call super() so the order of inheritance shouldn't matter here
class CombinedSchemaOpts(ja_schema.SchemaOpts, sql_schema.ModelSchemaOpts):
    pass

class ArtistSchema(ma.ModelSchema, Schema):
    OPTIONS_CLASS = CombinedSchemaOpts
    class Meta(ma.ModelSchema.Meta, Schema.Meta):
        model = models.Artist
        type_ = 'artists'

    albums = fields.Relationship(...)

[multimeric]

Actually @sloria I'm wondering if we could do some Metaclass magic where, whenever someone subclasses a Schema with multiple parents, we automatically create an Options class and a class Meta that combine the Options/Meta of the parent classes.

This would let us combine these two libraries with much less code.

[multimeric]

Another issue with this integration is when you have a relationship that doesn't have included data, e.g.

{'id': '2', 'type': 'users'}

What happens here is that the Relationship field serializes it to an ID (2), which is correct, because that's the only information we have access to. But then in the make_instance hook, we set user = 2, and because 2 is not a valid model object, it fails with:

AttributeError: 'int' object has no attribute '_sa_instance_state'

[multimeric]

My current solution to this problem is to use a custom Relationship field subclass that queries for the associated object:

class Relationship(BaseRelationship):
    def extract_value(self, data):
        ret = super().extract_value(data)
        if isinstance(ret, (int, str)) and hasattr(self.schema.opts, 'model') and hasattr(self.schema.opts, 'sqla_session'):
            return self.schema.opts.sqla_session.query(self.schema.opts.model).get(ret)
        return ret

Ideally we'd only set the foreign key field when this happens, (ie instead of setting user, we set user_id), but I don't think Marshmallow can do this.