UnicodeEncodeError: 'charmap' codec can't encode ... character maps to <undefined>

[zvpxz]

Traceback (most recent call last): File "scdl.py", line 408, in main() File "scdl.py", line 87, in main parse_url(arguments["-l"]) File "scdl.py", line 160, in parse_url download_user_tracks(item) File "scdl.py", line 228, in download_user_tracks download_track(track) File "scdl.py", line 322, in download_track print("Downloading " + title) File "C:\Python34\lib\encodings\cp437.py", line 19, in encode return codecs.charmap_encode(input,self.errors,encoding_map)[0] UnicodeEncodeError: 'charmap' codec can't encode characters in position 12-18: character maps to

I've tried downloading using the single and all-songs method to download. The song title is "ｊｕｎｇｌｅ" not jungle and I guess it's having trouble encoding those characters. I've read somewhere on overstock that python can use the latin-1 charmap? since it has most if not all of the characters? I've taken for understanding that it has more characters than UTF-8.

[flyingrub]

could you share the link please ?

[zvpxz]

Soundcloud link? https://soundcloud.com/robert-dingus/janguru-mizu or this one https://soundcloud.com/robert-dingus/jungle-jig

Thanks for checking up!

[flyingrub]

are you on windows ?

fly@Minotoor ~/Musique/test $ scdl -l https://soundcloud.com/robert-dingus/janguru-mizu
Soundcloud Downloader
Downloading to /home/fly/Musique/test...

Found a track
Downloading Ｊａｎｇｕｒｕ- ｍｉｚｕ
100% [......................................................] 2906069 / 2906069
Settings tags...

Ｊａｎｇｕｒｕ- ｍｉｚｕ.mp3 Downloaded.

[flyingrub]

@mauricebasement, @zvpxz could you test with the latest commit : https://github.com/flyingrub/scdl/commit/a0af030c26ea783c05b6a90996173a081452e514 if every char are working well on windows ?

[zvpxz]

I get this with the latest. Excuse me for being so promptly...

C:\scdl\scdl>python scdl.py -l https://soundcloud.com/robert-dingus/janguru-mizu

Traceback (most recent call last): File "scdl.py", line 35, in from scdl import version File "C:\scdl\scdl\scdl.py", line 35, in from scdl import version ImportError: cannot import name 'version'

C:\scdl\scdl>

[rhodessteve]

Similar sort of error here. Windows 8.1 US / Python 3.42 Track: https://soundcloud.com/user48736353001/7-f Name: 7 ∂ƒ∆ [rough mix]

Traceback (most recent call last): File "C:\Python34\Scripts\scdl-script.py", line 9, in load_entry_point('scdl==v1.1.0', 'console_scripts', 'scdl')() File "C:\Python34\lib\site-packages\scdl\scdl.py", line 113, in main parse_url(arguments["-l"]) File "C:\Python34\lib\site-packages\scdl\scdl.py", line 188, in parse_url download_all_user_tracks(item) File "C:\Python34\lib\site-packages\scdl\scdl.py", line 233, in download_all_user_tracks parse_url(this_url) File "C:\Python34\lib\site-packages\scdl\scdl.py", line 177, in parse_url download_track(item) File "C:\Python34\lib\site-packages\scdl\scdl.py", line 358, in download_track log("Downloading " + title, strverbosity=1) File "C:\Python34\lib\site-packages\scdl\scdl.py", line 65, in log print(colored(str, 'white')) File "C:\Python34\lib\encodings\cp437.py", line 19, in encode return codecs.charmap_encode(input,self.errors,encoding_map)[0] UnicodeEncodeError: 'charmap' codec can't encode character '\u2202' in position 19: character maps to

[flyingrub]

@zvpxz you need to install the package with pip, not launch the file itself.

@windowsUser : https://github.com/flyingrub/scdl/commit/435da05f09c2d6d62fedeb877a8c90627e5097c1 can you test the latest release on pypi ?

[rhodessteve]

Ok, i tried the new build. the good news is, it works with the unicode. The bad news is that: a) 7 ∂ƒ∆ [rough mix] became 7 Γêé╞ÆΓêå [rough mix] and 2 ∂ƒx 126b became 2 Γêé╞Æx 126b. b) I get ←[31mError trying to set the tags...←[0m for all mp3s (there are no tags in the mp3s) c) it sometimes crashes: Traceback (most recent call last): File "C:\Python34\Scripts\scdl-script.py", line 9, in load_entry_point('scdl==v1.1.1', 'console_scripts', 'scdl')() File "C:\Python34\lib\site-packages\scdl\scdl.py", line 113, in main parse_url(arguments["-l"]) File "C:\Python34\lib\site-packages\scdl\scdl.py", line 188, in parse_url download_all_user_tracks(item) File "C:\Python34\lib\site-packages\scdl\scdl.py", line 236, in download_all_user_tracks response = urllib.request.urlopen(url) File "C:\Python34\lib\urllib\request.py", line 153, in urlopen return opener.open(url, data, timeout) File "C:\Python34\lib\urllib\request.py", line 461, in open response = meth(req, response) File "C:\Python34\lib\urllib\request.py", line 571, in http_response 'http', request, response, code, msg, hdrs) File "C:\Python34\lib\urllib\request.py", line 499, in error return self._call_chain(_args) File "C:\Python34\lib\urllib\request.py", line 433, in _call_chain result = func(_args) File "C:\Python34\lib\urllib\request.py", line 579, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 500: Internal Server Error

I ran it a total of 3 times before it downloaded all 96 tracks for the user. First time it gave me 14, then 38. Many thanks for your work on this. At least it works eventually.

[flyingrub]

it works perfectly on unix ... i m not using windows so it's hard to me to fix those errors. If you could pastebin every different kind of error you had, it would help for this script to support windows

[zvpxz]

Pip works on Windows? Sorry for the late response. Like stated, the package works flawless off of Unix. I'll just stick to using it on my Mint partition.. All the hard work put in to this masterpiece is very much appreciated.

[flyingrub]

Don't know if pip work on windows, but you could have done a python3 setup.py install (at least this is the command for linux)

[rhodessteve]

Can confirm that pip works on Windows. Thats how i installed it, and updated the script each time. Not sure if the token gets wiped in the config file but I saved it anyway. Remaining problems stated above (tagging, weird test on unicode, sometimes HTTP 500 crash).

[flyingrub]

https://github.com/flyingrub/scdl/issues/129

[dweee]

Is there any way of getting around this issue apart from using Linux? I don't want to setup a Virtual Machine just for downloading playlists.

[mauricebasement]

You can help get rid of the issues by proper bug reporting, trying to fix bugs in the code by yourself. I cannot provide any support for windows as I see using it as the main issue. I suggest you start using linux in general, it's next level in anyways.

Am 30.03.2016 22:34 schrieb David:

Is there any way of getting around this issue apart from using Linux? I don't want to setup a Virtual Machine just for downloading playlists.

You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub [1]

*

Links:

[1] https://github.com/flyingrub/scdl/issues/37#issuecomment-203622589

[e2del]

Same error : https://soundcloud.com/infine-music/sets/bachar-mar-khalif-ya-balad-out

a possible fix (tested on windows):

in function : title = title.encode('utf8').decode('utf8') to replace : title = title.encode('utf-8', 'ignore').decode(sys.stdout.encoding) Windows cmd is not utf-8.

and open the playlist file as utf-8 with codecs with codecs.open(playlist_name + '.m3u', 'w+', 'utf-8') as playlist_file:

[flyingrub]

https://github.com/flyingrub/scdl/blob/master/scdl/scdl.py#L337 the title line is already like that. i fixed the playlist name https://github.com/flyingrub/scdl/commit/1c7fc643a17a23f7e3a77716e167ea2c4c7e9e1d. Any error now ?

[e2del]

sorry I do a error title = title.encode('utf-8', 'ignore').decode(sys.stdout.encoding) must be replaced by title = title.encode('utf8').decode('utf8')

the playlist name works great. the problem is 'sys.stdout.encoding' in cmd.exe return not utf8 (something bad)

[e2del]

the main problem is in write file function :

playlist_file.write(
            '#EXTINF:{0},{1}{3}{2}{3}'.format(
                duration, title, filename, os.linesep
            )

In windows the file writes in 'ascii' if not open as utf8

[e2del]

fix #145