search

ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
132.55k stars 10.05k forks source link

Udemy: HTTP Error 403: FORBIDDEN for video data (not JSON metadata) #16372

Closed c2d628ba98 closed 6 years ago

c2d628ba98 commented 6 years ago

What is the purpose of your issue?

I saw this issue #13074 and I can access JSON metadata but not the video file itself hosted on S3 bucket :

$ youtube-dl -u <an@example.com> --cookies ~/cookies.txt --write-sub -o "./%(playlist)s/%(chapter_number)s-%(chapter)s/%(autonumber)03d-%(title)s.%(ext)s" https://www.udemy.com/modern-javascript-from-the-beginning/ --verbose
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-u', u'PRIVATE', u'--cookies', u'cookies.txt', u'--write-sub', u'-o', u'./%(playlist)s/%(chapter_number)s-%(chapter)s/%(autonumber)03d-%(title)s.%(ext)s', u'https://www.udemy.com/modern-javascript-from-the-beginning/', u'--verbose']
Type account password and press [Return]: 
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.05.01
[debug] Python version 2.7.14 (CPython) - Linux-4.16.5-300.fc28.x86_64-x86_64-with-fedora-28-Twenty_Eight
[debug] exe versions: none
[debug] Proxy map: {}
[udemy:course] Downloading login popup
[udemy:course] modern-javascript-from-the-beginning: Downloading webpage
[udemy:course] 1463348: Downloading course curriculum
[download] Downloading playlist: Modern JavaScript From The Beginning
[udemy:course] playlist Modern JavaScript From The Beginning: Collected 119 video ids (downloading 119 of them)
[download] Downloading video 1 of 119
[udemy] Downloading login popup
[udemy] 8757064: Downloading webpage
[udemy] 8757064: Downloading lecture JSON
[debug] Default format spec: best/bestvideo+bestaudio
[debug] Invoking downloader on u'http://knowband-asset-bucket.s3.amazonaws.com/2017-12-08_21-29-56-f75d23f1264e4aa4226939e2aee2b718/WebHD_720p.mp4'
ERROR: unable to download video data: HTTP Error 403: Forbidden
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1900, in process_info
    success = dl(filename, info_dict)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1839, in dl
    return fd.download(name, info)
  File "/usr/local/bin/youtube-dl/youtube_dl/downloader/common.py", line 365, in download
    return self.real_download(filename, info_dict)
  File "/usr/local/bin/youtube-dl/youtube_dl/downloader/http.py", line 340, in real_download
    establish_connection()
  File "/usr/local/bin/youtube-dl/youtube_dl/downloader/http.py", line 109, in establish_connection
    ctx.data = self.ydl.urlopen(request)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2203, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib64/python2.7/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/usr/lib64/python2.7/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib64/python2.7/urllib2.py", line 473, in error
    return self._call_chain(*args)
  File "/usr/lib64/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/usr/lib64/python2.7/urllib2.py", line 556, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 403: Forbidden
dstftw commented 6 years ago

You must provide account credentials to reproduce the issue or another freely available video.

c2d628ba98 commented 6 years ago

@dstftw please check this one - https://www.udemy.com/javascript-essentials/

c2d628ba98 commented 6 years ago

I also found a link that works and it looks like the URL for video files is diffrent so perhpas they changed something?

$ youtube-dl -u <an@exmaple.com> --cookies ~/cookies.txt --write-sub -o "./%(playlist)s/%(chapter_number)s-%(chapter)s/%(autonumber)03d-%(title)s.%(ext)s" https://www.udemy.com/introduction-to-game-development-with-unity/ --verbose
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-u', u'PRIVATE', u'--cookies', u'cookies.txt', u'--write-sub', u'-o', u'./%(playlist)s/%(chapter_number)s-%(chapter)s/%(autonumber)03d-%(title)s.%(ext)s', u'https://www.udemy.com/introduction-to-game-development-with-unity/', u'--verbose']
Type account password and press [Return]: 
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.05.01
[debug] Python version 2.7.12 (CPython) - Linux-4.4.0-122-generic-x86_64-with-Ubuntu-16.04-xenial
[debug] exe versions: avconv 2.8.14-0ubuntu0.16.04.1, avprobe 2.8.14-0ubuntu0.16.04.1, ffmpeg 2.8.14-0ubuntu0.16.04.1, ffprobe 2.8.14-0ubuntu0.16.04.1
[debug] Proxy map: {}
[udemy:course] Downloading login popup
[udemy:course] introduction-to-game-development-with-unity: Downloading webpage
[udemy:course] 1036542: Enrolling in the course
[udemy:course] 1036542: Downloading course curriculum
[download] Downloading playlist: 1036542
[udemy:course] playlist 1036542: Collected 18 video ids (downloading 18 of them)
[download] Downloading video 1 of 18
[udemy] Downloading login popup
[udemy] 6180882: Downloading webpage
[udemy] 6180882: Downloading lecture JSON
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on u'https://udemy-assets-on-demand.udemy.com/2016-12-10_16-35-45-9d23369ec89c5a65326711ea80226c57/WebHD_720p.mp4?nva=20180505023047&download=True&filename=Get-to-know-your-instructor.mov&token=0517c785998eba59664be'
[download] Destination: ./1036542/1-Introduction/001-Get to know your instructor.mp4
[download] 100% of 14.84MiB in 00:01
...
cptflammin commented 6 years ago

Get exactly same issue with the link above and paid course as well. Did they enforce protection?

config

youtube-dl -v [debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: [u'-v'] [debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8 [debug] youtube-dl version 2018.05.01 [debug] Python version 2.7.15 (CPython) - Darwin-17.5.0-x86_64-i386-64bit [debug] exe versions: ffmpeg 4.0, ffprobe 4.0 [debug] Proxy map: {}

cmd

youtube-dl --cookies ./udemy-cookies.txt -o '%(playlist)s/%(chapter_number)s - %(chapter)s/%(playlist_index)s. %(title)s.%(ext)s' https://www.udemy.com/javascript-essentials/learn/v4

output

[udemy:course] javascript-essentials: Downloading webpage [udemy:course] 707962: Enrolling in the course [udemy:course] 707962: Downloading course curriculum ERROR: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

vesper8 commented 6 years ago

the videos are easily downloadable using chrome extensions.. so whatever they changed, the protection is trivial.. probably only requires a minor fix