SoundCloud Scraping problem (soundcloud-lib error ; HTTPError: HTTP Error 403: Forbidden)

[soheilpaper]

I am trying to scrap new music form sound loud and upload them via telegram group so, I asked here and by not finding answer I have tried to use another Python SoundCloud scraper module by below commands, which could run on this Colab page, too:

pip install soundcloud-lib

from sclib import SoundcloudAPI, Track, Playlist

api = SoundcloudAPI()
playlist = api.resolve('https://soundcloud.com/razhavaniazha-com')

assert type(playlist) is Playlist

for track in playlist.tracks:
    filename = f'./{track.artist} - {track.title}.mp3'
    with open(filename, 'wb+') as fp:
        track.write_mp3_to(fp)

but by running the above codes, get this error:

HTTPError                                 Traceback (most recent call last)
<ipython-input-6-023e08e56a22> in <module>()
      2 
      3 api = SoundcloudAPI()
----> 4 playlist = api.resolve('https://soundcloud.com/razhavaniazha-com')
      5 
      6 assert type(playlist) is Playlist

9 frames
/usr/lib/python3.7/urllib/request.py in http_error_default(self, req, fp, code, msg, hdrs)
    647 class HTTPDefaultErrorHandler(BaseHandler):
    648     def http_error_default(self, req, fp, code, msg, hdrs):
--> 649         raise HTTPError(req.full_url, code, msg, hdrs, fp)
    650 
    651 class HTTPRedirectHandler(BaseHandler):

HTTPError: HTTP Error 403: Forbidden

I have tried this code on my pc and I get the same error, I guess the SoundCloud require a client ID for doing the music scraping. But, I asked here to understand how to do it without the client ID?

Also asked here:

https://stackoverflow.com/questions/68368288/soundcloud-scraping-problem-soundcloud-lib-error-httperror-http-error-403-f Thanks.

[sekian]

I think the "HTTP Error 403: Forbidden" was fixed with the commit: 00b4572 For now, you can try installing the library from the repository instead of the pypi release: pip install git+https://github.com/3jackdaws/soundcloud-lib.git

[mechashok]

I am having this same issue.

I have tried reinstalling from the repository instead of the pypi release, with no luck.

Code:

`def downloadSoundCloudMP3(link):

api = SoundcloudAPI()  # never pass a Soundcloud client ID that did not come from this library

track = api.resolve(link)

assert type(track) is Track

download_Folder = download_Path.get()

filename = download_Folder + '/' + '{track.artist} - {track.title}.mp3'

print("Downloading: " + link + " as: " + '{track.artist} - {track.title}.mp3')
print("---------")
print("   ")

with open(filename, 'wb+') as fp:
    track.write_mp3_to(fp)`

Error message:

`Found SoundCloud link: https://soundcloud.com/geordiehumm/mizormac-bites-the-dust-drill-sensei-mashup-hummy-edit

Exception in Tkinter callback Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/tkinter/init.py", line 1892, in call return self.func(args) File "/Users/showell/PycharmProjects/pythonDownloader/Bulk MP3 Downloader.py", line 78, in start sortDownloadURLs(link) File "/Users/showell/PycharmProjects/pythonDownloader/Bulk MP3 Downloader.py", line 96, in sortDownloadURLs downloadSoundCloudMP3(link) File "/Users/showell/PycharmProjects/pythonDownloader/Bulk MP3 Downloader.py", line 131, in downloadSoundCloudMP3 track = api.resolve(link) File "/Users/showell/PycharmProjects/pythonDownloader/lib/python3.9/site-packages/sclib/sync.py", line 59, in resolve self.get_credentials() File "/Users/showell/PycharmProjects/pythonDownloader/lib/python3.9/site-packages/sclib/sync.py", line 54, in get_credentials js_text = f'{get_page(script)}' File "/Users/showell/PycharmProjects/pythonDownloader/lib/python3.9/site-packages/sclib/sync.py", line 14, in get_page return get_url(url).decode('utf-8') File "/Users/showell/PycharmProjects/pythonDownloader/lib/python3.9/site-packages/sclib/sync.py", line 11, in get_url return urlopen(url).read() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 214, in urlopen return opener.open(url, data, timeout) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 523, in open response = meth(req, response) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 632, in http_response response = self.parent.error( File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 561, in error return self._call_chain(args) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 494, in _call_chain result = func(*args) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 641, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden`

Has this issue been resolved anywhere else, or have SoundCloud changed their website again?

[DJoepie]

Are you sure you're using the most up-to-date version of this repo @mechashok ? Because for me it's still working.

Image of tool working


Another workaround is downloading the repo from github and move (and rename) the soundcloud-lib-master-folder to the root of your own project and import it from there.

from SCapi import Playlist, SoundcloudAPI, Track

Image of file structure of private project


[x-j]

Another workaround is downloading the repo from github and move (and rename) the soundcloud-lib-master-folder to the root of your own project and import it from there.

I can confirm this solution worked for me, but earlier I was also getting 403 Errors when running this code installed from pip. But it would be nice to not have to do this workaround, do you have the latest version of sclib uploaded to pip?

[DJoepie]

do you have the latest version of sclib uploaded to pip?

I believe only the owner of this repo can update the pip version.

[x-j]

Yeah, I meant to addres this to @3jackdaws

[thedtvn]

I am having this same issue.

I have tried reinstalling from the repository instead of the pypi release, with no luck.

Code:

`def downloadSoundCloudMP3(link):

api = SoundcloudAPI()  # never pass a Soundcloud client ID that did not come from this library

track = api.resolve(link)

assert type(track) is Track

download_Folder = download_Path.get()

filename = download_Folder + '/' + '{track.artist} - {track.title}.mp3'

print("Downloading: " + link + " as: " + '{track.artist} - {track.title}.mp3')
print("---------")
print("   ")

with open(filename, 'wb+') as fp:
    track.write_mp3_to(fp)`

Error message:

`Found SoundCloud link: https://soundcloud.com/geordiehumm/mizormac-bites-the-dust-drill-sensei-mashup-hummy-edit

Exception in Tkinter callback Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/tkinter/init.py", line 1892, in call return self.func(args) File "/Users/showell/PycharmProjects/pythonDownloader/Bulk MP3 Downloader.py", line 78, in start sortDownloadURLs(link) File "/Users/showell/PycharmProjects/pythonDownloader/Bulk MP3 Downloader.py", line 96, in sortDownloadURLs downloadSoundCloudMP3(link) File "/Users/showell/PycharmProjects/pythonDownloader/Bulk MP3 Downloader.py", line 131, in downloadSoundCloudMP3 track = api.resolve(link) File "/Users/showell/PycharmProjects/pythonDownloader/lib/python3.9/site-packages/sclib/sync.py", line 59, in resolve self.get_credentials() File "/Users/showell/PycharmProjects/pythonDownloader/lib/python3.9/site-packages/sclib/sync.py", line 54, in get_credentials js_text = f'{get_page(script)}' File "/Users/showell/PycharmProjects/pythonDownloader/lib/python3.9/site-packages/sclib/sync.py", line 14, in get_page return get_url(url).decode('utf-8') File "/Users/showell/PycharmProjects/pythonDownloader/lib/python3.9/site-packages/sclib/sync.py", line 11, in get_url return urlopen(url).read() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 214, in urlopen return opener.open(url, data, timeout) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 523, in open response = meth(req, response) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 632, in http_response response = self.parent.error( File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 561, in error return self._call_chain(args) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 494, in _call_chain result = func(*args) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 641, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden`

Has this issue been resolved anywhere else, or have SoundCloud changed their website again?

@mechashok yep them has change

[thedtvn]

some off my info

[thedtvn]

https://api-widget.soundcloud.com/resolve?url=https%3A//soundcloud.com/narx-mo/mameyudoufu-featranasol-monaremix&format=json&client_id=LBCcHmRB8XSStWL6wKH2HPACspQlXg2P&app_version=1663571498

• this can fix that but it use other client_id not user client_id
• fine at https://widget.sndcdn.com/widget-9-46c45ca2df38.js
• regex ,client_id:u?"(user_client_id)":"(widget_client_id)",

[thedtvn]

or use client_id at https://a-v2.sndcdn.com/assets/50-bef420db.js regex: {client_id:"(.*?)"}

[thedtvn]

and i has fix it at https://github.com/thedtvn/soundcloud-lib

[3jackdaws]

Hi @thedtvn, if you submit a PR with a fix for this, I will review and merge it.