search

Miserlou / SoundScrape

SoundCloud (and Bandcamp and Mixcloud) downloader in Python.
MIT License
1.42k stars 146 forks source link

Exception while decoding some soundcloud urls #169

Open jakubincloud opened 7 years ago

jakubincloud commented 7 years ago

Some of the soundcloud urls generate JSON exception

I think the response code for http call is 401, and some links have additional protection

Example links:

Traceback:

(soundscrape) ❯ soundscrape https://soundcloud.com/goldfishlive/ultra-miami-set-2017-live-stage-goldfish
Traceback (most recent call last):
  File "/Users/jakubzygmunt/.virtualenvs/soundscrape/bin/soundscrape", line 11, in <module>
    sys.exit(main())
  File "/Users/jakubzygmunt/.virtualenvs/soundscrape/lib/python2.7/site-packages/soundscrape/soundscrape.py", line 119, in main
    process_soundcloud(vargs)
  File "/Users/jakubzygmunt/.virtualenvs/soundscrape/lib/python2.7/site-packages/soundscrape/soundscrape.py", line 198, in process_soundcloud
    hard_track_url = get_hard_track_url(item_id)
  File "/Users/jakubzygmunt/.virtualenvs/soundscrape/lib/python2.7/site-packages/soundscrape/soundscrape.py", line 518, in get_hard_track_url
    json_response = response.json()
  File "/Users/jakubzygmunt/.virtualenvs/soundscrape/lib/python2.7/site-packages/requests/models.py", line 866, in json
    return complexjson.loads(self.text, **kwargs)
  File "/Users/jakubzygmunt/.virtualenvs/soundscrape/lib/python2.7/site-packages/simplejson/__init__.py", line 516, in loads
    return _default_decoder.decode(s)
  File "/Users/jakubzygmunt/.virtualenvs/soundscrape/lib/python2.7/site-packages/simplejson/decoder.py", line 374, in decode
    obj, end = self.raw_decode(s)
  File "/Users/jakubzygmunt/.virtualenvs/soundscrape/lib/python2.7/site-packages/simplejson/decoder.py", line 404, in raw_decode
    return self.scan_once(s, idx=_w(s, idx).end())
simplejson.scanner.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
TGPSKI commented 7 years ago

Haha - ran into the same with https://soundcloud.com/lane8music/lane-8-spring-2017-mixtape! Such a good mixtape!!

Also this guy: https://soundcloud.com/snbrn/spring-sessions

ghost commented 7 years ago

I was going to comment this, but you already did. I'm trying to bulk convert a bunch of soundcloud links from a file, and this is the main thing stopping me. The other little thing is the failure to convert links to songs inside of sets, but that's not too hard to fix by doing some link parsing. If the issue above got fixed I think I could actually work with this tool. Seems neat, but it's a bit inconsistent.

jakubincloud commented 7 years ago

@TGPSKI try to find it on youtube and use https://rg3.github.io/youtube-dl/ 👍

grantmerola commented 7 years ago

I believe this issue is the result of either a 403 or 401 response and is the same issue as #166 and #156 . There are various reasons why the sever is returning these codes. For example youtube-dl test in its Soundcloud file for songs that can not be streamed in that case from the US which is how I came across this issue.

I have tested the other bad links on here and they do not have the same error but a similar error

# not streamable in US 
/Users/gm/Downloads- : !soundscrape https://soundcloud.com/the-concept-band/sets/goldrushed-2013-album
#added print statement of response line 505 or 518 in soundscrape.py
<Response [401]>
Traceback (most recent call last):
  File "/usr/local/bin/soundscrape", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/site-packages/soundscrape/soundscrape.py", line 119, in main
    process_soundcloud(vargs)
  File "/usr/local/lib/python3.6/site-packages/soundscrape/soundscrape.py", line 250, in process_soundcloud
    tracks = get_soundcloud_api_playlist_data(resolved.id)['tracks']
  File "/usr/local/lib/python3.6/site-packages/soundscrape/soundscrape.py", line 507, in get_soundcloud_api_playlist_data
    parsed = response.json()
  File "/usr/local/lib/python3.6/site-packages/requests/models.py", line 894, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/simplejson/__init__.py", line 516, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python3.6/site-packages/simplejson/decoder.py", line 370, in decode
    obj, end = self.raw_decode(s)
  File "/usr/local/lib/python3.6/site-packages/simplejson/decoder.py", line 400, in raw_decode
    return self.scan_once(s, idx=_w(s, idx).end())
simplejson.scanner.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

/Users/gm/Downloads- : !soundscrape https://soundcloud.com/goldfishlive/ultra-miami-set-2017-live-stage-goldfish
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/soundscrape/soundscrape.py", line 184, in process_soundcloud
    resolved = client.get('/resolve', url=artist_url, limit=200)
  File "/usr/local/lib/python3.6/site-packages/soundcloud/client.py", line 133, in _request
    return wrapped_resource(make_request(method, url, kwargs))
  File "/usr/local/lib/python3.6/site-packages/soundcloud/request.py", line 148, in make_request
    result.raise_for_status()
  File "/usr/local/lib/python3.6/site-packages/requests/models.py", line 937, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://api.soundcloud.com/tracks/315269564?client_id=175c043157ffae2c6d5fed16c3d95a4c

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/soundscrape", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/site-packages/soundscrape/soundscrape.py", line 119, in main
    process_soundcloud(vargs)
  File "/usr/local/lib/python3.6/site-packages/soundscrape/soundscrape.py", line 198, in process_soundcloud
    hard_track_url = get_hard_track_url(item_id)
  File "/usr/local/lib/python3.6/site-packages/soundscrape/soundscrape.py", line 519, in get_hard_track_url
    json_response = response.json()
  File "/usr/local/lib/python3.6/site-packages/requests/models.py", line 894, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/simplejson/__init__.py", line 516, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python3.6/site-packages/simplejson/decoder.py", line 370, in decode
    obj, end = self.raw_decode(s)
  File "/usr/local/lib/python3.6/site-packages/simplejson/decoder.py", line 400, in raw_decode
    return self.scan_once(s, idx=_w(s, idx).end())
simplejson.scanner.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

/Users/gm/Downloads- : !soundscrape https://soundcloud.com/lane8music/lane-8-spring-2017-mixtape
#only changed line
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://api.soundcloud.com/tracks/314737276?client_id=175c043157ffae2c6d5fed16c3d95a4c

/Users/gm/Downloads- : !soundscrape https://soundcloud.com/solomun/bbc-1-essential-mix-live-live-from-solomun1-at-pacha-ibiza
#added print statement of response line 505 or 518 in soundscrape.py
<Response [401]>
#only changed line
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://api.soundcloud.com/tracks/293283192?client_id=175c043157ffae2c6d5fed16c3d95a4c

/Users/gm/Downloads- : !soundscrape https://soundcloud.com/snbrn/spring-sessions
#added print statement of response line 505 or 518 in soundscrape.py
<Response [401]>
#only changed line
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://api.soundcloud.com/tracks/316259408?client_id=175c043157ffae2c6d5fed16c3d95a4c

/Users/gm/Downloads- : !soundscrape https://soundcloud.com/celldweller/electric-eye
#added print statement of response line 505 or 518 in soundscrape.py
<Response [401]>
#only changed line
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://api.soundcloud.com/tracks/308914769?client_id=175c043157ffae2c6d5fed16c3d95a4c

full log log.txt

jonathan00 commented 7 years ago

same for me with https://soundcloud.com/cosmicgateofficial/cosmic-gate-live-at-tomorrowland-2017

jonathan00 commented 7 years ago

works: youtube-dl https://soundcloud.com/cosmicgateofficial/cosmic-gate-live-at-tomorrowland-2017

ceuskervin commented 6 years ago

any update on this issue? I'd like to use only soundscrape, but will give this youtube-dl a try if no progress..