hotkit / django-async

A simple asynchronous execution Django application with proper database transaction management
http://www.kirit.com/Django%20Async
Boost Software License 1.0
35 stars 20 forks source link

flush-queue fails if unprintable char in args #30

Open tcarlander opened 5 years ago

tcarlander commented 5 years ago

if the args contain some non ascii chars sometimes flush-queue stops with:

  File "/....../lib/python3.4/site-packages/async/management/commands/flush_queue.py", line 63, in run
    print ("%s: %s" % (job.id, job))
UnicodeEncodeError: 'ascii' codec can't encode character '\xe9' in position 137: ordinal not in range(128)

the args in the above failure where: ["fiche-de-suivi-mensuel-collines", "ben", {"query": "l\u00e9a", "group": null, "status": null}, {"host": "ben.mdca.wfp.org", "is_secure": false}, false, false]

the job finished but was never marked completed as it broke on print job

KayEss commented 5 years ago

Python and Unicode. Gotta love it! This is Python 3? I thought that was supposed to fix Python's terrible Unicode support. :-1: For 2 I would suggest trying

print (u"%s: %s" % (job.id, job))

But I don't know what the right thing for 3 is....

tcarlander commented 5 years ago

It is for python 3. I have looked more in to it and realise that the issue is in the unicode() of the job somewhere here:

class Job(models.Model):
.
.
.
.
    def __unicode__(self):
        # __unicode__: Instance of 'bool' has no 'items' member
        # pylint: disable=E1103
        arglist = simplejson.loads(self.args)
        arglist = [repr(s) for s in arglist]
        kwargs = simplejson.loads(self.kwargs)
        kwargs = sorted([u"%s=%s" % (k, repr(v)) for k, v in kwargs.items()])
        args = u', '.join(arglist + kwargs)

it occurs when doing the print of the job

KayEss commented 5 years ago

Hmm, if I had to point fingers I think it'd be at the repr(s) in the arglist, although all of the strings there are marked Unicode.