Unicode error

[labrys]

Sab from source: f76e2a7b5607686bea7a3fb3ffcf8c2a403cde12 Windows 10x64 Python 2.7.13x64

2017-06-20 20:52:12,720::INFO::[postproc:549] Traceback:
Traceback (most recent call last):
  File "H:\usr\local\sabnzbd\sabnzbd\postproc.py", line 351, in process_job
    complete_dir = file_sorter.detect(dirname, complete_dir)
  File "H:\usr\local\sabnzbd\sabnzbd\tvsort.py", line 107, in detect
    complete_dir = self.sorter.get_final_path()
  File "H:\usr\local\sabnzbd\sabnzbd\tvsort.py", line 220, in get_final_path
    path = self.construct_path()
  File "H:\usr\local\sabnzbd\sabnzbd\tvsort.py", line 360, in construct_path
    path = path_subst(sorter, mapping)
  File "H:\usr\local\sabnzbd\sabnzbd\tvsort.py", line 898, in path_subst
    return ''.join(newpath)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xf6 in position 5: invalid start byte

[Safihre]

What was the job name and what was the Sorting string you were applying?

[sanderjo]

UnicodeDecodeError: 'utf8' codec can't decode byte 0xf6 in position 5: invalid start byte

0xF6 ... so probably ö (o - umlaut) in path or file name. Correct?

This can happen in SABnzbd due to a mix of Unicode (Sab) and old-school Windows-encoding (in downloads). Hard / impossible to solve.

[thezoggy]

it just looks like tvsort doesnt support unicode from what he pasted, as ''.join(newpath) would be for string only.. should be changed to u''.join(newpath)

[Safihre]

Paging @shypike?

[shypike]

@Safihre It looks like the "newpath" list is a mix of Unicode and Latin-1, which means that this is the only solution: u''.join([unicoder(x) for x in newpath])

[Safihre]

@labrys For testing, do you have the (file)name of the post?

[labrys]

I don't think I have the filename any more, but I don't recall it having any non-ascii characters in it. As for sorting, I was either sorting by size ascending, sorting alphabetically ascending, or (if filtering counts, which seems unlikely) filtering by part of a series name as those are the three actions I use regularly.

[labrys]

Also, not sure if its related, but I do get a lot of these too when I sort: With tracebacks such as:

2017-06-22 17:08:14,872::ERROR::[__init__:899] Saving \\?\G:\tmp\nzb\downloads\Veep.S06E03.1080p.WEBRip.x264-MOROSE\__ADMIN__\SABnzbd_nzo_00fhbo failed
2017-06-22 17:08:14,877::INFO::[__init__:900] Traceback:
Traceback (most recent call last):
  File "H:\usr\local\sabnzbd\sabnzbd\__init__.py", line 885, in save_data
    with open(path, 'wb') as data_file:
IOError: [Errno 2] No such file or directory: u'\\\\?\\G:\\tmp\\nzb\\downloads\\Veep.S06E03.1080p.WEBRip.x264-MOROSE\\__ADMIN__\\SABnzbd_nzo_00fhbo'

[thezoggy]

the true error is that it didnt save 2017-06-22 17:08:14,872::ERROR::[__init__:899] Saving \\?\G:\tmp\nzb\downloads\Veep.S06E03.1080p.WEBRip.x264-MOROSE\__ADMIN__\SABnzbd_nzo_00fhbo failed

it shouldnt have tried to even open the file afterwards

[Safihre]

@labrys we are taking about different sorting, not sorting of the queue but sorting as in Config > Sorting. So as a post processing step after downloading. What have you set up there?

The data saving/loading problem I have to investigate, it's unrelated and due to my other speed improvement changes.

[labrys]

Rest are disabled. Also if it matters, TV category items default to the series subfolder