search

ArchiveTeam / terroroftinytown

URLTeam's second generation of URL shortener archiving tools
http://urlte.am
MIT License
71 stars 15 forks source link

upload_result: UnicodeDecodeError: 'unicodeescape' codec can't decode byte 0x5c in position 7: \ at end of string #11

Closed chfoo closed 9 years ago

chfoo commented 10 years ago 
INFO:terroroftinytown.client.tracker:Uploading to tracker.
Traceback (most recent call last):
  File "scraper.py", line 60, in <module>
    main()
  File "scraper.py", line 42, in main
    result)
  File "scraper.py", line 48, in try_with_tracker
    return func(*args, **kwargs)
  File "/home/chris/data-chris-aspire-5741/Documents/programming/terroroftinytown-client-grab/terroroftinytown/terroroftinytown/client/tracker.py", line 27, in wrapper
    return func(*args, **kwargs)
  File "/home/chris/data-chris-aspire-5741/Documents/programming/terroroftinytown-client-grab/terroroftinytown/terroroftinytown/client/tracker.py", line 76, in upload_item
    'results': json.dumps(results, encoding='unicode-escape'),
  File "/usr/lib/python2.7/json/__init__.py", line 250, in dumps
    sort_keys=sort_keys, **kw).encode(obj)
  File "/usr/lib/python2.7/json/encoder.py", line 207, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python2.7/json/encoder.py", line 270, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python2.7/json/encoder.py", line 233, in _encoder
    o = o.decode(_encoding)
UnicodeDecodeError: 'unicodeescape' codec can't decode byte 0x5c in position 7: \ at end of string
Process RunScraper returned exit code 1 for Item

Looks like result of #4. Python 2 json behaves differently from Python 3. Need to make json.dumps() return a unicode string with JSON \u escapes instead of using unicodeescape and encode it ourselves with Latin-1.