Apispec 6.1.0: Accessing API docs throws TypeError: Object of type Decimal is not JSON serializable

[kaibr]

The update to 6.1.0 causes an 500 internal server error for me when trying to view my API docs in a browser. Using flask-smorest. No such error in 6.0.2.

Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 2213, in call return self.wsgi_app(environ, start_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 2193, in wsgi_app response = self.handle_exception(e) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/flask_cors/extension.py", line 165, in wrapped_function return cors_after_request(app.make_response(f(*args, *kwargs))) ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 2190, in wsgi_app response = self.full_dispatch_request() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1486, in full_dispatch_request rv = self.handle_user_exception(e) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/flask_cors/extension.py", line 165, in wrapped_function return cors_after_request(app.make_response(f(args, kwargs))) ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1484, in full_dispatch_request rv = self.dispatch_request() ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1469, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(view_args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/flask_smorest/spec/init.py", line 131, in _openapi_json json.dumps(self.spec.to_dict(), indent=2), mimetype="application/json" File "/usr/local/lib/python3.11/json/init.py", line 238, in dumps **kw).encode(obj) ^^^^^^^^^^^ File "/usr/local/lib/python3.11/json/encoder.py", line 202, in encode chunks = list(chunks) ^^^^^^^^^^^^ File "/usr/local/lib/python3.11/json/encoder.py", line 432, in _iterencode yield from _iterencode_dict(o, _current_indent_level) File "/usr/local/lib/python3.11/json/encoder.py", line 406, in _iterencode_dict yield from chunks File "/usr/local/lib/python3.11/json/encoder.py", line 406, in _iterencode_dict yield from chunks File "/usr/local/lib/python3.11/json/encoder.py", line 406, in _iterencode_dict yield from chunks [Previous line repeated 3 more times] File "/usr/local/lib/python3.11/json/encoder.py", line 439, in _iterencode o = _default(o) ^^^^^^^^^^^ File "/usr/local/lib/python3.11/json/encoder.py", line 180, in default raise TypeError(f'Object of type {o.class.name} ' TypeError: Object of type Decimal is not JSON serializable

Package Version Editable project location

---

apispec 6.1.0 Flask 2.3.2 flask-smorest 0.42.0

[lafrech]

From the error, it looks like you're using the Decimal field without specifying as_string=True.

See warning here: https://marshmallow.readthedocs.io/en/stable/marshmallow.fields.html#marshmallow.fields.Decimal.

6.1.0 introduces a fix specifically to use the field to serialize min/max values to avoid such a 500 when those values are not JSON serializable by standard json lib. I guess you're passing the min as int or float so before the fix it would work, but since the fix it is serialized as Decimal. Passing as_string=True should do the trick.

I'm surprised you don't get errors when JSON serializing your API output, though, so maybe I'm wrong, but this should get you on the right track.

[kaibr]

You're right, I use the Decimal field, without as_string=True and with a range validator to which I pass min as a float.

I also set app.json to a custom flask.json.provider.JSONProvider which serializes Decimal as float. That's why I'm not getting errors serializing API output.

My assumption was that that the API docs generation would use that same JSONProvider. That assumption seems to be incorrect? If so, how can I make the docs generation use this JSONProvider?

Thanks for any help!

class DecimalJSONEncoder(json.JSONEncoder):
    """Encodes Decimal as float."""

    def default(self, object):
        if isinstance(object, decimal.Decimal):
            return float(object)
        return super().default(object)

class CustomJsonProvider(JSONProvider):
    def dumps(self, obj, **kwargs):
        return json.dumps(obj, **kwargs, cls=DecimalJSONEncoder)

    def loads(self, s: str | bytes, **kwargs):
        return json.loads(s, **kwargs)

def create_app(...):
    ...
    app = flask.Flask(app_name)
    app.json = CustomJsonProvider(app)
    ...
    flask_api = flask_smorest.Api(app)

[lafrech]

I've been going back and forth in the past about using flask.json or standard json to serialize stuff (payload, docs).

Your use case makes me think we should always use flask.json.

Would you like to try your code base with this branch: https://github.com/marshmallow-code/flask-smorest/tree/flask_json.

If you confirm it works, we could add non-reg tests and ship.

[kaibr]

I can confirm that the code in the flask_json branch works for me. Newest apispec (6.3.0) and no exception viewing the API docs.

[lafrech]

@kaibr, I've been trying to add a test, see https://github.com/marshmallow-code/flask-smorest/tree/flask_json.

Any idea why this doesn't work? The custom serializer doesn't seem to be called. Am I not passing it correctly?

[lafrech]

Fix released in 0.44.

Note that Flask default JSON serializer now serialises Decimal so the custom decimal encoder shown above in not needed anymore.