KeyError: 'release_year' on recommended playlists

[xenithorb]

$ sc-dl --url https://soundcloud.com/twofeetmusic/quick-musical-doodles-and-sex/recommended
Connecting ...
Downloading 5 related tracks
Traceback (most recent call last):
  File "/home/xenith/virtualenvs/soundcloud_dl/bin/sc-dl", line 11, in <module>
    sys.exit(main())
  File "/home/xenith/virtualenvs/soundcloud_dl/lib/python2.7/site-packages/soundcloud_dl/soundcloud_dl.py", line 33, in main
    downloaderObject.Download()
  File "/home/xenith/virtualenvs/soundcloud_dl/lib/python2.7/site-packages/soundcloud_dl/downloader/downloader.py", line 342, in Download
    self.getRecommendedTracks(data, no_tracks)
  File "/home/xenith/virtualenvs/soundcloud_dl/lib/python2.7/site-packages/soundcloud_dl/downloader/downloader.py", line 147, in getRecommendedTracks
    self.getSingleTrack(track)
  File "/home/xenith/virtualenvs/soundcloud_dl/lib/python2.7/site-packages/soundcloud_dl/downloader/downloader.py", line 70, in getSingleTrack
    'year': track['release_year'].encode('utf-8') if track['release_year'] else '',
KeyError: 'release_year

[suyashb95]

Fixed here, anything else?

[xenithorb]

$ python ~/virtualenvs/soundcloud_dl/soundcloud-dl/soundcloud_dl.py --url https://soundcloud.com/twofeetmusic --all
Connecting ...
User profile found.
Saving in : /home/xenith/ownCloud/Music/SoundCloud/Two Feet
Traceback (most recent call last):
  File "/home/xenith/virtualenvs/soundcloud_dl/soundcloud-dl/soundcloud_dl.py", line 39, in <module>
    main()
  File "/home/xenith/virtualenvs/soundcloud_dl/soundcloud-dl/soundcloud_dl.py", line 33, in main
    downloaderObject.Download()
  File "/home/xenith/virtualenvs/soundcloud_dl/soundcloud-dl/downloader/downloader.py", line 332, in Download
    self.getUploadedTracks(data)
  File "/home/xenith/virtualenvs/soundcloud_dl/soundcloud-dl/downloader/downloader.py", line 137, in getUploadedTracks
    self.getSingleTrack(track)
  File "/home/xenith/virtualenvs/soundcloud_dl/soundcloud-dl/downloader/downloader.py", line 70, in getSingleTrack
    'year': track.get('release_year', '').encode('utf-8'),
AttributeError: 'NoneType' object has no attribute 'encode'

[xenithorb]

I fixed this to get things working locally by:

--- a/downloader/downloader.py
+++ b/downloader/downloader.py
@@ -65,10 +65,10 @@ class soundcloudDownloader(object):
         if isinstance(track, resource.Resource):
             track = track.fields()
         metadata = {
-            'title':track.get('title', '').encode('utf-8'),
-            'artist':track['user']['username'].encode('utf-8'),
-            'year': track.get('release_year', '').encode('utf-8'),
-            'genre': track.get('genre', '').encode('utf-8')
+            'title':track.get('title', ''),
+            'artist':track['user']['username'],
+            'year': track.get('release_year', ''),
+            'genre': track.get('genre', '')
         }
         if track['downloadable']:
             try:

But I don't normally work with python, which really means I'm not certain of the implications of not converting it to UTF-8. (Or maybe it's implicit? I don't know. :)

[suyashb95]

That's what I did as well. Haha unicode is a nightmare, I'm learning about its caveats in Python.

[xenithorb]

--- a/downloader/downloader.py
+++ b/downloader/downloader.py
@@ -65,10 +65,10 @@ class soundcloudDownloader(object):
         if isinstance(track, resource.Resource):
             track = track.fields()
         metadata = {
-            'title':track.get('title', '').encode('utf-8'),
-            'artist':track['user']['username'].encode('utf-8'),
-            'year': track.get('release_year', '').encode('utf-8'),
-            'genre': track.get('genre', '').encode('utf-8')
+            'title': str(track.get('title', '')).encode('utf-8'),
+            'artist': str(track['user']['username']).encode('utf-8'),
+            'year': str(track.get('release_year', '')).encode('utf-8'),
+            'genre': str(track.get('genre', '')).encode('utf-8')
         }
         if track['downloadable']:
             try:

This works too, and I think it retains the original intent of using encode() to deal with UTF-8

[xenithorb]

Woops, messed up, change this from my last:

-            'artist': str(track['user']['username']).encode('utf-8'),
+            'artist': track['user']['username'].encode('utf-8'),

[xenithorb]

It's still not perfect though, one of the problems I always end up with this script is if I do high limit it eventually runs into something it can't handle:

Connecting to stream...
Response: 200
File Size: 2.96 MB
Saving as: ¥VOU - Praxeis.mp3
[####################]  | 100.00 %
Download complete.
Traceback (most recent call last):
  File "/home/xenith/virtualenvs/soundcloud_dl/soundcloud-dl/soundcloud_dl.py", line 39, in <module>
    main()
  File "/home/xenith/virtualenvs/soundcloud_dl/soundcloud-dl/soundcloud_dl.py", line 33, in main
    downloaderObject.Download()
  File "/home/xenith/virtualenvs/soundcloud_dl/soundcloud-dl/downloader/downloader.py", line 342, in Download
    self.getRecommendedTracks(data, no_tracks)
  File "/home/xenith/virtualenvs/soundcloud_dl/soundcloud-dl/downloader/downloader.py", line 147, in getRecommendedTracks
    self.getSingleTrack(track)
  File "/home/xenith/virtualenvs/soundcloud_dl/soundcloud-dl/downloader/downloader.py", line 97, in getSingleTrack
    self.tagFile(new_filename, metadata, track['artwork_url'])
  File "/home/xenith/virtualenvs/soundcloud_dl/soundcloud-dl/downloader/downloader.py", line 256, in tagFile
    data=image
  File "/home/xenith/virtualenvs/soundcloud_dl/lib/python2.7/site-packages/mutagen/id3/_frames.py", line 68, in __init__
    kwargs.get(checker.name, checker.default))
  File "/home/xenith/virtualenvs/soundcloud_dl/lib/python2.7/site-packages/mutagen/id3/_frames.py", line 78, in __setattr__
    self._setattr(name, checker.validate(self, value))
  File "/home/xenith/virtualenvs/soundcloud_dl/lib/python2.7/site-packages/mutagen/id3/_specs.py", line 428, in validate
    raise TypeError
TypeError

[suyashb95]

High limit as in? Setting the limit higher than the number of songs there are?

[xenithorb]

Not exactly, I don't think that's related to the limit I just think it has to do with the fact that people like to put dumb characters in their song names. (and that I inevitably run across one) You can see there it's failing after it's downloaded and while it's tagging it.

[suyashb95]

Oh well, there must be something wrong with the image tagging. The JSON from soundcloud has None in places so python treats it as a NoneType object, typecasting to str should work

[xenithorb]

@@ -253,7 +253,7 @@ class soundcloudDownloader(object):
                     mime='image/jpeg',
                     type=3,
                     desc=u'Cover',
-                    data=image
+                    data=str(image)
                     )
                 )
             audio.tags["TIT2"] = TIT2(encoding=3, text=unicode(metadata.get('title', '').decode('utf-8')))

This does in fact stop it from failing, but interestingly it only populated the Title tag and the Genre with the Artist name. Must be a weird JSON payload or something

[suyashb95]

I just checked for the image and then set the tag, turns out in that particular track there is no artwork URL.

[suyashb95]

Not sure if converting the image to an str will work always, it's supposed to be a file. I've just added one more condition to check if the image file exists.