Soundcloud results in 403 error

ghost commented 7 years ago

Make sure you are using the latest version: run `youtube-dl --version` and ensure your version is 2017.01.02. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

[x] I've verified and I assure that I'm running youtube-dl 2017.01.02

Before submitting an issue make sure you have:

[x] At least skimmed through README and most notably FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[x] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

Verbose output:

$ youtube-dl -v https://soundcloud.com/efkv/hair
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'https://soundcloud.com/efkv/hair']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.01.02
[debug] Python version 2.7.9 - Linux-4.5.2-armada375-armv7l-with-Debian-8
[debug] exe versions: ffmpeg 2.8.9, ffprobe 2.8.9
[debug] Proxy map: {}
[soundcloud] efkv/hair: Resolving id
[soundcloud] efkv/hair: Downloading info JSON
ERROR: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/home/rafa/bin/youtube-dl/youtube_dl/extractor/common.py", line 406, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/home/rafa/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2001, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python2.7/urllib2.py", line 437, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 550, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 475, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 558, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)

SwadicalRag commented 7 years ago

I'm having the same issue, +1

EDIT: replacing my local binary with an installation from pip seems to have fixed this

weird

yan12125 commented 7 years ago

@Rafa652 Could you paste the output of:

curl -v "http://api.soundcloud.com/resolve.json?url=http://soundcloud.com/efkv/hair&client_id=fDoItMDbsbZz8dY16ZzARCZmzgHBPotA"

ghost commented 7 years ago

Sure thing. This is what I got:

* Hostname was NOT found in DNS cache
*   Trying 93.184.220.127...
* Connected to api.soundcloud.com (93.184.220.127) port 80 (#0)
> GET /resolve.json?url=http://soundcloud.com/efkv/hair&client_id=fDoItMDbsbZz8dY16ZzARCZmzgHBPotA HTTP/1.1
> User-Agent: curl/7.38.0
> Host: api.soundcloud.com
> Accept: */*
>
< HTTP/1.1 403 Forbidden
< Content-Type: */*
< Date: Wed, 04 Jan 2017 22:55:46 GMT
* Server ECS (lhr/47F2) is not blacklisted
< Server: ECS (lhr/47F2)
< Content-Length: 345
<
<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
         "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
        <head>
                <title>403 - Forbidden</title>
        </head>
        <body>
                <h1>403 - Forbidden</h1>
        </body>
</html>
* Connection #0 to host api.soundcloud.com left intact

yan12125 commented 7 years ago

Seems soundcloud no longer uses resolve.json

Berndwl commented 7 years ago

Does this happen on your home network or on a vps / etc? Soundcloud seems to blacklist ISPs linked to VPS providers ( atleast this happened to me with an OVH vps

resolve.json is still a valid call to the soundcloud api

ghost commented 7 years ago

@Berndwl I hadn't even thought of trying that. It looks like that's the problem. I was trying it on servers hosted by OVH and Linode, but just now when I used it on my machine it worked just fine. I suppose there's not much that can be done about it. Thanks though.

royalaid commented 7 years ago

It seem to be that Soundcloud followed in the footsteps of Netflix and implemented similar filtering to them and their VPN blocking :(.

Perflyst commented 6 years ago

Has anything else happened here? Works again.

I am using the latest GitHub release

$ youtube-dl -v https://soundcloud.com/efkv/hair
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'https://soundcloud.com/efkv/hair']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.06.19
[debug] Python version 2.7.12 (CPython) - Linux-4.15.0-23-generic-x86_64-with-neon-16.04-xenial
[debug] exe versions: avconv 2.8.14-0ubuntu0.16.04.1, avprobe 2.8.14-0ubuntu0.16.04.1, ffmpeg 2.8.14-0ubuntu0.16.04.1, ffprobe 2.8.14-0ubuntu0.16.04.1, rtmpdump 2.4
[debug] Proxy map: {}
[soundcloud] efkv/hair: Resolving id
[soundcloud] efkv/hair: Downloading info JSON
[soundcloud] 258086978: Downloading track url
[soundcloud] 258086978: Downloading m3u8 information
[soundcloud] 258086978: Downloading m3u8 information
[soundcloud] 258086978: Checking hls_opus_64_url video format URL
[soundcloud] 258086978: Checking hls_mp3_128_url video format URL
[soundcloud] 258086978: Checking http_mp3_128_url video format URL
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on u'https://cf-media.sndcdn.com/hNkwAhEkWoT5.128.mp3?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiKjovL2NmLW1lZGlhLnNuZGNkbi5jb20vaE5rd0FoRWtXb1Q1LjEyOC5tcDMiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE1Mjk4NDM0NTd9fX1dfQ__&Signature=hiZU45mCIzByOdJ3mQ-mUioGE5khrHhOf4QT6eqGAiPrBpIscfjKD1PZdOIDw-tN4RsVM~cau0ptCYGPU5s3DQpiR2T-A8ttImPEkukTVrdj8Qr1kOARfzLcn8Zk9wzBxun5QD1iEaQr4RKTU6zEp7~Ks0E9LMK3okml548s0fUUKuB5S-cHStVSgDzHXnkXO6~hxstGNcMiKUUCuqzEHCT7bfLFsywxMTo2ToR6~3ZvuLeU1WYeP6ch-gpeC9zHUaVaqctftrXVToU9aAWTyHl0gJ9R~BkX~dPoSJPKirh9uwYZAIgk5gE36Hwz6g1HS8efrauf03wp9wqsF6ryiw__&Key-Pair-Id=APKAJAGZ7VMH2PFPW6UQ'
[download] Destination: Hair in the Drain (dl in description)-258086978.mp3
[download]  45.2% of 3.69MiB at 259.68KiB/s ETA 00:07^C
ERROR: Interrupted by user

akx commented 6 years ago

This can also occur if the media is not available in the your IP address's country.

ytdl-org / youtube-dl