search

ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
131.91k stars 10k forks source link

[Udemy] Course download does not work on some courses and returns 403 forbidden after passing the publically available preview videos #28877

Closed FerasArabiat closed 2 years ago

FerasArabiat commented 3 years ago

Checklist

Verbose log

user1@user1 udemy-dl % youtube-dl --cookies /Users/user1/Downloads/udemy.com_cookies.txt -f 'best'  https://www.udemy.com/neo4j-foundations/ -o '%(playlist)s/%(chapter_number)s. %(chapter)s/%(playlist_index)s. %(title)s.%(ext)s' -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--cookies', '/Users/user1/Downloads/udemy.com_cookies.txt', '-f', 'best', 'https://www.udemy.com/neo4j-foundations/', '-o', '%(playlist)s/%(chapter_number)s. %(chapter)s/%(playlist_index)s. %(title)s.%(ext)s', '-v']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.04.26
[debug] Python version 3.9.4 (CPython) - macOS-10.15.7-x86_64-i386-64bit
[debug] exe versions: ffmpeg 4.3.1, ffprobe 4.3.1, rtmpdump 2.4
[debug] Proxy map: {}
[udemy:course] neo4j-foundations: Downloading webpage
[udemy:course] 692416: Downloading course curriculum
[download] Downloading playlist: 692416
[udemy:course] playlist 692416: Collected 76 video ids (downloading 76 of them)
[download] Downloading video 1 of 76
[udemy] 18178652: Downloading webpage
[udemy] 18178652: Downloading lecture JSON
[udemy] 22522866: Downloading m3u8 information
[debug] Invoking downloader on 'https://www.udemy.com/assets/22522866/files/2020-02-25_04-32-21-c81c9df6373bae728c6ad4e11306017d/1/hls/AVC_1920x1080_1200k_AAC-HE_64k/aa00837488289822bc8bd10f1e3dc6acf0d2.m3u8?provider=verizon&token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJwYXRoIjoiMjAyMC0wMi0yNV8wNC0zMi0yMS1jODFjOWRmNjM3M2JhZTcyOGM2YWQ0ZTExMzA2MDE3ZC8xLyIsImV4cCI6MTYxOTQ0Njk0Nn0.UH34mMtW_dNuh8HKBnbVxTbMtn5sppAPPKuPkuYaC3Y&v=1'
[download] 692416/1. Welcome to the course/01. Welcome.mp4 has already been downloaded
[download] 100% of 8.51MiB
[debug] ffmpeg command line: ffprobe -show_streams 'file:692416/1. Welcome to the course/01. Welcome.mp4'
[ffmpeg] Fixing malformed AAC bitstream in "692416/1. Welcome to the course/01. Welcome.mp4"
[debug] ffmpeg command line: ffmpeg -y -loglevel repeat+info -i 'file:692416/1. Welcome to the course/01. Welcome.mp4' -c copy -f mp4 -bsf:a aac_adtstoasc 'file:692416/1. Welcome to the course/01. Welcome.temp.mp4'
[download] Downloading video 2 of 76
[udemy] 4156842: Downloading webpage
[udemy] 4156842: Downloading lecture JSON
[udemy] 22655032: Downloading m3u8 information
[debug] Invoking downloader on 'https://www.udemy.com/assets/22655032/files/2020-03-04_02-05-12-212aaaffa14f70b3af7a7f4a0a94da61/1/hls/AVC_1920x1080_1200k_AAC-HE_64k/aa00b174d6874f6fb5231f5d7250de8d6670.m3u8?provider=verizon&token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE2MTk0NDY5NDksInBhdGgiOiIyMDIwLTAzLTA0XzAyLTA1LTEyLTIxMmFhYWZmYTE0ZjcwYjNhZjdhN2Y0YTBhOTRkYTYxLzEvIn0.Mna27tzeyEYSbQOO1qAHyCTaA9glYXf2tNjq4KRrQMo&v=1'
[download] 692416/1. Welcome to the course/02. Course Overview.mp4 has already been downloaded
[download] 100% of 13.86MiB
[debug] ffmpeg command line: ffprobe -show_streams 'file:692416/1. Welcome to the course/02. Course Overview.mp4'
[ffmpeg] Fixing malformed AAC bitstream in "692416/1. Welcome to the course/02. Course Overview.mp4"
[debug] ffmpeg command line: ffmpeg -y -loglevel repeat+info -i 'file:692416/1. Welcome to the course/02. Course Overview.mp4' -c copy -f mp4 -bsf:a aac_adtstoasc 'file:692416/1. Welcome to the course/02. Course Overview.temp.mp4'
[download] Downloading video 3 of 76
[udemy] 4156846: Downloading webpage
[udemy] 4156846: Downloading lecture JSON
[udemy] 22519730: Downloading m3u8 information
[debug] Invoking downloader on 'https://www.udemy.com/assets/22519730/files/2020-02-24_23-53-23-a806ae0cd151554115ac5ffe7dec1a41/1/hls/AVC_1920x1080_800k_AAC-HE_64k/aa000afee21e6278b65683de1eac381e8d83.m3u8?provider=verizon&token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJwYXRoIjoiMjAyMC0wMi0yNF8yMy01My0yMy1hODA2YWUwY2QxNTE1NTQxMTVhYzVmZmU3ZGVjMWE0MS8xLyIsImV4cCI6MTYxOTQ0Njk1Mn0.pi7JvnpYLfxgK4KtsMYyT3tzLRhw9bFuTkZHpgw8u78&v=1'
[download] 692416/1. Welcome to the course/03. What is a Graph.mp4 has already been downloaded
[download] 100% of 9.75MiB
[debug] ffmpeg command line: ffprobe -show_streams 'file:692416/1. Welcome to the course/03. What is a Graph.mp4'
[ffmpeg] Fixing malformed AAC bitstream in "692416/1. Welcome to the course/03. What is a Graph.mp4"
[debug] ffmpeg command line: ffmpeg -y -loglevel repeat+info -i 'file:692416/1. Welcome to the course/03. What is a Graph.mp4' -c copy -f mp4 -bsf:a aac_adtstoasc 'file:692416/1. Welcome to the course/03. What is a Graph.temp.mp4'
[download] Downloading video 4 of 76
[udemy] 5160978: Downloading webpage
[udemy] 5160978: Downloading lecture JSON
[debug] Invoking downloader on 'https://knowband-asset-bucket.s3.amazonaws.com/2020-02-24_23-14-43-4b84b17b8a746b328e2861e663d5a32c/WebHD_720p.mp4'
ERROR: unable to download video data: HTTP Error 403: Forbidden
Traceback (most recent call last):
  File "/usr/local/Cellar/youtube-dl/2021.4.26/libexec/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 1967, in process_info
    success = dl(filename, info_dict)
  File "/usr/local/Cellar/youtube-dl/2021.4.26/libexec/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 1906, in dl
    return fd.download(name, info)
  File "/usr/local/Cellar/youtube-dl/2021.4.26/libexec/lib/python3.9/site-packages/youtube_dl/downloader/common.py", line 366, in download
    return self.real_download(filename, info_dict)
  File "/usr/local/Cellar/youtube-dl/2021.4.26/libexec/lib/python3.9/site-packages/youtube_dl/downloader/http.py", line 351, in real_download
    establish_connection()
  File "/usr/local/Cellar/youtube-dl/2021.4.26/libexec/lib/python3.9/site-packages/youtube_dl/downloader/http.py", line 116, in establish_connection
    raise err
  File "/usr/local/Cellar/youtube-dl/2021.4.26/libexec/lib/python3.9/site-packages/youtube_dl/downloader/http.py", line 110, in establish_connection
    ctx.data = self.ydl.urlopen(request)
  File "/usr/local/Cellar/youtube-dl/2021.4.26/libexec/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 2279, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/local/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 523, in open
    response = meth(req, response)
  File "/usr/local/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 632, in http_response
    response = self.parent.error(
  File "/usr/local/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 561, in error
    return self._call_chain(*args)
  File "/usr/local/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 494, in _call_chain
    result = func(*args)
  File "/usr/local/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 641, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

Description

When trying to downloading some courses from Udemy, the downloads do not proceed passed the publically available previews.

I have tried these two and it did not work: https://www.udemy.com/course/neo4j-foundations/ https://www.udemy.com/course/the-ultimate-mysql-bootcamp-go-from-sql-beginner-to-expert/

Note the URL that the content is being pulled from differs between the working and non working URLS

[debug] Invoking downloader on 'https://www.udemy.com/assets/22519730/files/2020-02-24_23-53-23-a806ae0cd151554115ac5ffe7dec1a41/1/hls/AVC_1920x1080_800k_AAC-HE_64k/aa000afee21e6278b65683de1eac381e8d83.m3u8?provider=verizon&token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJwYXRoIjoiMjAyMC0wMi0yNF8yMy01My0yMy1hODA2YWUwY2QxNTE1NTQxMTVhYzVmZmU3ZGVjMWE0MS8xLyIsImV4cCI6MTYxOTQ0Njk1Mn0.pi7JvnpYLfxgK4KtsMYyT3tzLRhw9bFuTkZHpgw8u78&v=1'

vs

[debug] Invoking downloader on 'https://knowband-asset-bucket.s3.amazonaws.com/2020-02-24_23-14-43-4b84b17b8a746b328e2861e663d5a32c/WebHD_720p.mp4'

I can provide credentials as needed via email or lastpass whatever works best.

petr-hybler commented 3 years ago

same here.... any update in the foreseeable future?

Pontiac76 commented 3 years ago

Having the same problem, and using YTDL 2021-05-16

The command I'm using is the following (Removing my creds and pword from the CLI)

youtube-dl -u USER -p PASSWORD -o "./%(playlist)s/%(chapter_number)s - %(chapter)s/%(title)s.%(ext)s" "https://www.udemy.com/course/blendertutorial/"

[debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: ['-u', 'PRIVATE', '-p', 'PRIVATE', '-o', './%(playlist)s/%(chapter_number)s - %(chapter)s/%(title)s.%(ext)s', 'https://www.udemy.com/course/blendertutorial/', '--verbose'] [debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252 [debug] youtube-dl version 2021.05.16 [debug] Python version 3.4.4 (CPython) - Windows-10-10.0.19041 [debug] exe versions: none [debug] Proxy map: {} [udemy:course] Downloading login popup ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmp8rlhijs0\build\youtube_dl\extractor\common.py", line 634, in _request_webpage File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmp8rlhijs0\build\youtube_dl\YoutubeDL.py", line 2288, in urlopen File "C:\Python\Python34\lib\urllib\request.py", line 470, in open File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response File "C:\Python\Python34\lib\urllib\request.py", line 508, in error File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default

G:\udemy>youtube-dl -U youtube-dl is up-to-date (2021.05.16)

ThomasChr commented 3 years ago

Can confirm. It's only a problem when a video is hosted on AWS. The other videos (even some of the same course!) are working just fine.

J4bber commented 3 years ago

Udemy changed their DRM late April. i am only guessing here, but I bet that is the issue. Other downloaders that used to work for Udemy are also not working...

If anyone figures out how to make this work again, it would be greatly appreciated.

shaneholloman commented 3 years ago

Confirming issue trying so save for offline use

PS C:\Users\shane\Downloads\Network Plus> youtube-dl -U
youtube-dl is up-to-date (2021.06.06)
PS C:\Users\shane\Downloads\Network Plus> youtube-dl --cookies udemy.com_cookies.txt https://www.udemy.com/course/comptia-network-plus-video-training/learn/lecture/26720716#overview
[udemy:course] course: Downloading webpage
[udemy:course] 3985912: Downloading course curriculum
[download] Downloading playlist: 3985912
[udemy:course] playlist 3985912: Collected 189 video ids (downloading 189 of them)
[download] Downloading video 1 of 189
[udemy] 26720576: Downloading webpage
[udemy] 26720576: Downloading lecture JSON
[udemy] 26720576: Downloading lecture JSON
ERROR: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
PS C:\Users\shane\Downloads\Network Plus>
mohgabr commented 3 years ago

Any idea how to fix or bypass this issue?

greenbea commented 3 years ago

Same here having this issue.

shaneholloman commented 3 years ago

Any idea how to fix or bypass this issue?

@mohgabr For now I'm using this on courses I own https://github.com/FaisalUmair/udemy-downloader-gui < tremendous app, it's a gui, I know, but it works : )

smaga38 commented 3 years ago

Thanks to @jadence who said the following:
I was having the same 403 error despite purchasing the course. To fix it I did two things:

  1. Clear my udemy cookies, log back into udemy.com, and then export a new cookies.txt
  2. Changed the course URL from https://www.udemy.com/course// to https://www.udemy.com//

After the above two steps running youtube-dl --cookies cookies.txt https://www.udemy.com// --verbose finally worked!

johseg commented 3 years ago 
2. Changed the course URL from https://www.udemy.com/course/<course_name>/ to https://www.udemy.com/<course_name>/

that's the trick, thank you, works for me

mohgabr commented 3 years ago

@johseg @smaga38 I did both steps and didn't work for me which python version are you using?

youtube-dl --cookies cookies.txt https://naspers.udemy.com/protocol-buffers/ --verbose
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--cookies', u'cookies.txt', u'https://naspers.udemy.com/protocol-buffers/', u'--verbose']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.06.06
[debug] Python version 2.7.16 (CPython) - Darwin-20.5.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 4.4, ffprobe 4.4
[debug] Proxy map: {}
[udemy:course] protocol-buffers: Downloading webpage
[udemy:course] 1612492: Downloading course curriculum
ERROR: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 634, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2288, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 473, in error
    return self._call_chain(*args)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 556, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)

and same error using python3

python3 /usr/local/bin/youtube-dl --cookies cookies.txt --verbose https://naspers.udemy.com/protocol-buffers/
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--cookies', 'cookies.txt', '--verbose', 'https://naspers.udemy.com/protocol-buffers/']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.06.06
[debug] Python version 3.9.5 (CPython) - macOS-11.4-x86_64-i386-64bit
[debug] exe versions: ffmpeg 4.4, ffprobe 4.4
[debug] Proxy map: {}
[udemy:course] protocol-buffers: Downloading webpage
[udemy:course] 1612492: Downloading course curriculum
ERROR: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 634, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2288, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/local/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 523, in open
    response = meth(req, response)
  File "/usr/local/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 632, in http_response
    response = self.parent.error(
  File "/usr/local/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 561, in error
    return self._call_chain(*args)
  File "/usr/local/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 494, in _call_chain
    result = func(*args)
  File "/usr/local/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 641, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
johseg commented 3 years ago

yeah also have the same issue again :( It works for some files but not for others (as stated by Thomas above)

dirkf commented 2 years ago

See:

29410

29705

22306

Also see why a dedicated tool for Udemy was abandoned.