liuliqiang / redisbeat

celery redis scheduler, dynamic add/modify/delete task from celery.
MIT License
173 stars 44 forks source link

jsonpickle issues parsing datetime #42

Open NoahCardoza opened 7 months ago

NoahCardoza commented 7 months ago

I noticed this projects requirements.txt file lists Celery 4 and I'm using Celery 5 which is probably the root of this bug. Unfortunately I cannot downgrade Celery since I'm using Python 3.11 and Celery 4 depends on an older version of vine which wasn't upgraded yet to deal with the removal of inspect.formatargspec. I'm not exactly sure what is causing the error, but I found adding the following jsonpickle handler fixed the issue.

# https://stackoverflow.com/a/35634809/6169961
class DatePickleISO8601(jsonpickle.handlers.DatetimeHandler):
    def flatten(self, obj, data):
        pickler = self.context
        if not pickler.unpicklable:
            return str(obj)
        cls, args = obj.__reduce__()
        flatten = pickler.flatten
        payload = obj.isoformat()
        args = [payload] + [flatten(i, reset=False) for i in args[1:]]
        data["__reduce__"] = (flatten(cls, reset=False), args)
        return data

    def restore(self, data):
        cls, args = data["__reduce__"]
        unpickler = self.context
        restore = unpickler.restore
        cls = restore(cls, reset=False)
        value = datetime.fromisoformat(args[0])
        return value

jsonpickle.handlers.registry.register(datetime, DatePickleISO8601)

Hopefully this saves someone else a few hours. If you guys are open to PR's, I'd be happy to throw one together.

liuliqiang commented 7 months ago

Hi @NoahCardoza I'm apology to hear that.

Yes, I'm welcome for the PR, if you can help it, please help to create one.

Much thanks.

aabrodskiy commented 7 months ago

Thank you @NoahCardoza, you saved me a few hours of troubleshooting on this one! Having exact same issue.

@liuliqiang it's a great project, thank you for maintaining it! Might it be possible to bump requirements and support celery 5 and latest jsonpickle (3.0.2 and above)?