Unicode handling issue while downloading track

[bong0]

I got this Error today when running the script:

TMPDIR=/media/hdd/nosync timeout 43140 sc-syncer -u b0ngo -c $MY_CLIENT_ID -o /media/hdd/soundcloud_mirror -w -r >/dev/null
Traceback (most recent call last):
  File /usr/local/lib/python3.2/dist-packages/ssyncer/strack.py, line 183, in download
    self.get(id)))
UnicodeEncodeError: 'ascii' codec can't encode character '\xe4' in position 14: ordinal not in range(128)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File /usr/local/bin/sc-syncer, line 115, in <module>
    main()
  File /usr/local/bin/sc-syncer, line 100, in main
    limit)
  File /usr/local/bin/sc-syncer, line 35, in downloader
    if strack.download(output) is False:
  File /usr/local/lib/python3.2/dist-packages/ssyncer/strack.py, line 186, in download
    os.remove(local_file)
OSError: [Errno 2] No such file or directory: '/media/hdd/soundcloud_mirror/alohastone/181672091-tasschen-kuchen-ein-aloha-mixtape-fur-den-progolog-adventskalender'

Running it a second time (after also having upgraded to the last version), only the first issue is reported:

Traceback (most recent call last):
  File "/usr/local/bin/sc-syncer", line 128, in <module>
    main()
  File "/usr/local/bin/sc-syncer", line 112, in main
    args.limit)
  File "/usr/local/bin/sc-syncer", line 35, in downloader
    if strack.download(output, max_retry) is False:
  File "/usr/local/lib/python3.2/dist-packages/ssyncer/strack.py", line 181, in download
    print("\nDownloading %s (%d).." % (self.get("title"), self.get("id")))
UnicodeEncodeError: 'ascii' codec can't encode character '\xe4' in position 14: ordinal not in range(128)

It's obviously about this https://soundcloud.com/alohastone/tasschen-kuchen-ein-aloha-mixtape-fur-den-progolog-adventskalender track

[Sliim]

Hi bong0! Hmm I can't reproduce this issue on my system :(. Which locales are you using?

Thx :)

[bong0]

$ env | egrep 'LANG|LC_CTYPE'
LANG=en_GB.UTF-8
LC_CTYPE=en_US.utf8

/etc/locale.gen: en_GB.UTF-8 UTF-8

[Sliim]

Hi! I pushed a change on the master branch (8489bd) that should fix this issue. Can you try it? I havn't the capability to reproduce this error here..

Thx!

[bong0]

I wasn't sure whether the pip package is current because builds are failing at travis, so I checked it out from git and did a sudo python setup.py install. Output is the same as before :/

TMPDIR=/media/hdd/nosync timeout 43140 python3 /usr/local/bin/sc-syncer -u b0ngo -c $MY_CLIENT_ID -o /media/hdd/soundcloud_mirror -w -r
Traceback (most recent call last):
  File "/usr/local/bin/sc-syncer", line 128, in <module>
    main()
  File "/usr/local/bin/sc-syncer", line 112, in main
    args.limit)
  File "/usr/local/bin/sc-syncer", line 35, in downloader
    if strack.download(output, max_retry) is False:
  File "/usr/local/lib/python3.2/dist-packages/ssyncer/strack.py", line 181, in download
    print("\nDownloading %s (%d).." % (self.get("title"), self.get("id")))
UnicodeEncodeError: 'ascii' codec can't encode character '\xe4' in position 14: ordinal not in range(128)

[Sliim]

Hmm ok, I reverted my changes!

I think this is a locales issue on your system, you don't have en_US.UTF-8 in your /etc/locale.gen, so it's not installed.

Can you try to do: LC_CTYPE=en_GB.UTF-8 sc-syncer .... to confirm that is the cause? or add en_US.UTF-8 in your /etc/locale.gen

I run the script with LC_ALL=en_GB.UTF-8 envvar and I get this error too (en_GB.UTF-8 is not on my system).

[Sliim]

Hmm I just see that your LC_CTYPEenvvar seems incorrect.. (en_US.utf8 instead of en_US.utf-8)

[bong0]

@Sliim Your hint was good. I don't know how this happened but rasbian probably ships with the the GB locale preset. This is a problem when it's not also generated by locale.gen. I now activated en_GB and others in /etc/locale.gen and the issue disappeared.

Sorry for bothering :/

[Sliim]

No problem :)