datetime.date(2008, 5, 27) is not JSON serializable

[n3storm]

I get this error while requesting a list of resources:

´´´ 127.0.0.1 - - [23/Dec/2013 14:11:58] "GET /entities HTTP/1.1" 500 - Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1836, in call return self.wsgi_app(environ, start_response) File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1820, in wsgi_app response = self.make_response(self.handle_exception(e)) File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1403, in handle_exception reraise(exc_type, exc_value, tb) File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1817, in wsgi_app response = self.full_dispatch_request() File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1477, in full_dispatch_request rv = self.handle_user_exception(e) File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1381, in handle_user_exception reraise(exc_type, exc_value, tb) File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1475, in full_dispatch_request rv = self.dispatch_request() File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1461, in dispatch_request return self.view_functionsrule.endpoint File "/usr/local/lib/python2.7/dist-packages/sandman/sandman.py", line 397, in get_collection return collection_response(resources) File "/usr/local/lib/python2.7/dist-packages/sandman/sandman.py", line 228, in collection_response return _collection_json_response(resources) File "/usr/local/lib/python2.7/dist-packages/sandman/sandman.py", line 101, in _collection_json_response return jsonify(resources=result_list) File "/usr/local/lib/python2.7/dist-packages/flask/json.py", line 238, in jsonify indent=indent), File "/usr/local/lib/python2.7/dist-packages/flask/json.py", line 126, in dumps rv = _json.dumps(obj, kwargs) File "/usr/share/pyshared/simplejson/init.py", line 334, in dumps kw).encode(obj) File "/usr/share/pyshared/simplejson/encoder.py", line 239, in encode chunks = list(chunks) File "/usr/share/pyshared/simplejson/encoder.py", line 551, in _iterencode for chunk in _iterencode_dict(o, _current_indent_level): File "/usr/share/pyshared/simplejson/encoder.py", line 514, in _iterencode_dict for chunk in chunks: File "/usr/share/pyshared/simplejson/encoder.py", line 422, in _iterencode_list for chunk in chunks: File "/usr/share/pyshared/simplejson/encoder.py", line 514, in _iterencode_dict for chunk in chunks: File "/usr/share/pyshared/simplejson/encoder.py", line 561, in _iterencode o = _default(o) File "/usr/local/lib/python2.7/dist-packages/flask/json.py", line 83, in default return _json.JSONEncoder.default(self, o) File "/usr/share/pyshared/simplejson/encoder.py", line 213, in default raise TypeError(repr(o) + " is not JSON serializable") TypeError: datetime.date(2008, 5, 27) is not JSON serializable ´´´

Database field is of datetime type. Listing an editing through admin interface works ok.

Cheers.

[jeffknupp]

Let me quickly create a test database to recreate the issue. You wouldn't happen to have your database creation SQL lying around anywhere, would you?

[n3storm]

Its a 356 fields table (glups) with real customer data 13 954 rows (not that much).

If you I agree I will try to replicate with a smaller version of the table and see what happens.

[jeffknupp]

No, that's fine. I'll create my own on a local mysql instance. Out of curiosity, what database engine were you using?

[jeffknupp]

I've found that this usually happens when a "DateTime" column is actually only storing a "Date" (i.e. MM-DD-YYYY instead of MM-DD-YYYY HH:mm:SS). If that's the case in your scenario, it looks like a deficiency of the SQLAlchemy introspection portion. Could you check if that's the case.

Regardless, I'll open a ticket with them, as this should be supported (if the databases support it, which they do)

[n3storm]

That's right, all rows in that column are storing MM-DD-YYYY and it fails since first row! If I add HH:mm:SS to the cell, it fails on next row so it's almost confirmed you are right.

[dolinsky]

Not sure if this has been figured out yet or not, but I encountered the same issue (we have both date and datetime columns in our PG dbs). The answer was in the flask.json_encoder class, which extends the simplejson encoder and adds support for datetime, but not date. I overrode the default method with the following:

#!/usr/bin/env python

from sandman import app
from flask.json import JSONEncoder

class CustomJSONEncoder(JSONEncoder):
    def default(self, obj):
        try:
            if isinstance(obj, date):
                return obj.isoformat()
            iterable = iter(obj)
        except TypeError:
            pass
        else:
            return list(iterable)
        return JSONEncoder.default(self, obj)

app.json_encoder = CustomJSONEncoder

https://gist.github.com/dolinsky/9124542

[jeffknupp]

Thanks for the info. That may mean the solution is as simple as not using Flask's jsonify and just using the builtin json.dumps instead.

[mritunjay10]

Not sure if this has been figured out yet or not, but I encountered the same issue (we have both date and datetime columns in our PG dbs). The answer was in the flask.json_encoder class, which extends the simplejson encoder and adds support for datetime, but not date. I overrode the default method with the following:

#!/usr/bin/env python

from sandman import app
from flask.json import JSONEncoder

class CustomJSONEncoder(JSONEncoder):
    def default(self, obj):
        try:
            if isinstance(obj, date):
                return obj.isoformat()
            iterable = iter(obj)
        except TypeError:
            pass
        else:
            return list(iterable)
        return JSONEncoder.default(self, obj)

app.json_encoder = CustomJSONEncoder

https://gist.github.com/dolinsky/9124542

Thanks, this worked for me!