Unable to download JSON metadata: HTTP Error 403: Forbidden on Udemy

Checklist

[ ] I'm reporting a broken site support issue
[x ] I've verified that I'm running youtube-dl version 2019.09.01
[x ] I've checked that all provided URLs are alive and playable in a browser
[ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x ] I've searched the bugtracker for similar bug reports including closed ones
[ ] I've read bugs section in FAQ

Verbose log

E:\>youtube-dl -U
youtube-dl is up-to-date (2019.09.01)

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', '-u', 'PRIVATE', '-p', 'PRIVATE', '-F', '-o', '\\Test\\%(playlist)s/%(chapter_number)s - %(chapter)s/%(playlist_index)s. %(title)s
.%(ext)s', 'https://www.udemy.com/course/linux-bash-shell-scripting-complete-guide-incl-awk-sed/']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2019.09.01
[debug] Python version 3.4.4 (CPython) - Windows-7-6.1.7601-SP1
[debug] exe versions: none
[debug] Proxy map: {}
[udemy:course] Downloading login popup
[udemy:course] Logging in
[udemy:course] course: Downloading webpage
[udemy:course] 969354: Downloading course curriculum
[download] Downloading playlist: 969354
[udemy:course] playlist 969354: Collected 94 video ids (downloading 94 of them)
[download] Downloading video 1 of 94
[udemy] Downloading login popup
[udemy] 14300420: Downloading webpage
[udemy] 14300420: Downloading lecture JSON
[udemy] 14300420: Downloading lecture JSON
ERROR: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the
 latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpbzyg5d3a\build\youtube_dl\extractor\common.py", line 627, in _request_webpage
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpbzyg5d3a\build\youtube_dl\YoutubeDL.py", line 2229, in urlopen
  File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
  File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
  File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
  File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
  File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', '-u', 'PRIVATE', '-p', 'PRIVATE', '-F', '-o', '%(playlist)s/%(chapter_number)s - %(chapter)s/%(playlist_index)s. %(title)s.%(ext)s
', 'https://www.udemy.com/the-data-science-course-complete-data-science-bootcamp/']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2019.09.01
[debug] Python version 3.4.4 (CPython) - Windows-7-6.1.7601-SP1
[debug] exe versions: none
[debug] Proxy map: {}
[udemy:course] Downloading login popup
[udemy:course] Logging in
[udemy:course] the-data-science-course-complete-data-science-bootcamp: Downloading webpage
[udemy:course] 1754098: Downloading course curriculum
[download] Downloading playlist: 1754098
[udemy:course] playlist 1754098: Collected 381 video ids (downloading 381 of them)
[download] Downloading video 1 of 381
[udemy] Downloading login popup
[udemy] 10799982: Downloading webpage
[udemy] 10799982: Downloading lecture JSON
[udemy] 13201104: Downloading m3u8 information
[info] Available formats for 13201104:
format code  extension  resolution note
144p         mp4        256x144     201k , libx264@ 139k, 30fps, aac  @ 62k (44100Hz)
hls-259      mp4        256x144     259k , avc1.4d400c, 30.0fps, mp4a.40.2
360p         mp4        640x360     571k , libx264@ 446k, 30fps, aac  @125k (44100Hz)
hls-670      mp4        640x360     670k , avc1.4d401e, 30.0fps, mp4a.40.2
480p         mp4        854x480     927k , libx264@ 802k, 30fps, aac  @125k (44100Hz)
hls-1061     mp4        854x480    1061k , avc1.4d401f, 30.0fps, mp4a.40.2
720p         mp4        1280x720   1304k , libx264@1179k, 30fps, aac  @125k (44100Hz)
hls-1469     mp4        1280x720   1469k , avc1.4d401f, 30.0fps, mp4a.40.2
hls-2853     mp4        1920x1080  2853k , avc1.4d4028, 30.0fps, mp4a.40.2 (best)
[download] Downloading video 2 of 381
[udemy] 10762626: Downloading webpage
[udemy] 10762626: Downloading lecture JSON

ERROR: Interrupted by user

Description

I'm getting Unable to download JSON metadata for https://www.udemy.com/course/linux-bash-shell-scripting-complete-guide-incl-awk-sed/

I've tried a similar script for another course in Udemy as shown above where the metadata is downloaded and the video resolution files are shown. I'm using the latest version of youtube-dl.

Please advice if you can provide a resolution for this issue. If you require user authentication details, I can provide the same in a separate email rather than in this public issue board.

jason@jason-HP-Compaq-Pro-6300-All-in-One-PC:~/Videos/education/CSS-The Complete Guide$ sudo youtube-dl -U
youtube-dl is up-to-date (2019.09.01)
jason@jason-HP-Compaq-Pro-6300-All-in-One-PC:~/Videos/education/CSS-The Complete Guide$ youtube-dl --cookies cookies.txt -o "%(playlist)s/%(chapter_number)s - %(chapter)s/%(playlist_index)s. %(title)s.%(ext)s" https://www.udemy.com/course/css-the-complete-guide-incl-flexbox-grid-sass/ --playlist-end 23 -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--cookies', u'cookies.txt', u'-o', u'%(playlist)s/%(chapter_number)s - %(chapter)s/%(playlist_index)s. %(title)s.%(ext)s', u'https://www.udemy.com/course/css-the-complete-guide-incl-flexbox-grid-sass/', u'--playlist-end', u'23', u'-v']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.09.01
[debug] Python version 2.7.15+ (CPython) - Linux-5.0.0-25-generic-x86_64-with-Ubuntu-18.04-bionic
[debug] exe versions: ffmpeg 3.4.6, ffprobe 3.4.6
[debug] Proxy map: {}
[udemy:course] course: Downloading webpage
[udemy:course] 1561458: Downloading course curriculum
[download] Downloading playlist: 1561458
[udemy:course] playlist 1561458: Collected 266 video ids (downloading 23 of them)
[download] Downloading video 1 of 23
[udemy] 9429312: Downloading webpage
[udemy] 9429312: Downloading lecture JSON
[udemy] 9429312: Downloading lecture JSON
ERROR: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 627, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2229, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python2.7/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 473, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)

I'm getting the same issue trying to download a different course: https://www.udemy.com/course/css-the-complete-guide-incl-flexbox-grid-sass/

I'm using Ubuntu and using the cookie approach.

Strangely, I was able to start downloading a different course: https://www.udemy.com/android-development-java-android-studio-masterclass/

I am happy to provide more information if necessary.

$ lsb_release -a 
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.3 LTS
Release:    18.04
Codename:   bionic

$ youtube-dl --version
2019.09.01
$ youtube-dl --verbose --cookies=cookies.txt "https://www.udemy.com/course/learn-visual-studio-code/learn/"
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', '--cookies=cookies.txt', 'https://www.udemy.com/course/learn-visual-studio-code/learn/']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.09.01
[debug] Python version 3.6.8 (CPython) - Linux-4.15.0-60-generic-x86_64-with-Ubuntu-18.04-bionic
[debug] exe versions: ffmpeg 3.4.6, ffprobe 3.4.6
[debug] Proxy map: {}
[udemy:course] course: Downloading webpage
[udemy:course] 2022242: Downloading course curriculum
[download] Downloading playlist: 2022242
[udemy:course] playlist 2022242: Collected 42 video ids (downloading 42 of them)
[download] Downloading video 1 of 42
[udemy] 12524810: Downloading webpage
[udemy] 12524810: Downloading lecture JSON
[udemy] 12524810: Downloading lecture JSON
ERROR: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/home/micki/.local/lib/python3.6/site-packages/youtube_dl/extractor/common.py", line 627, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/home/micki/.local/lib/python3.6/site-packages/youtube_dl/YoutubeDL.py", line 2229, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python3.6/urllib/request.py", line 532, in open
    response = meth(req, response)
  File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.6/urllib/request.py", line 570, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 650, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)

Same issue here with different course.

Same issue with pluralsight-dl script using youtube-dl and my username and password! it worked till 14 days ago as i checked ...

[pluralsight:course] csharp-fundamentals: Downloading JSON metadata [download] Downloading playlist: Accelerated C# Fundamentals [pluralsight:course] playlist Accelerated C# Fundamentals: Collected 111 video ids (downloading 111 of them) [download] Downloading video 1 of 111 [pluralsight] Downloading login page [pluralsight] Logging in ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

I also got the same error with an Udemy course.

Unable to download webpage: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. File "/usr/local/lib/python2.7/dist-packages/youtube_dl/extractor/common.py", line 627, in _request_webpage return self._downloader.urlopen(url_or_request) File "/usr/local/lib/python2.7/dist-packages/youtube_dl/YoutubeDL.py", line 2229, in urlopen return self._opener.open(req, timeout=self._socket_timeout) File "/usr/lib/python2.7/urllib2.py", line 435, in open response = meth(req, response) File "/usr/lib/python2.7/urllib2.py", line 548, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python2.7/urllib2.py", line 473, in error return self._call_chain(args) File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain result = func(args) File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)

Same with me too.... from several days...unable to download from UDEMY.

'[debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: ['--username', 'PRIVATE', '--password', 'PRIVATE', '--verbose', '-o', 'E:/Udemy/%(playlist)s/%(chapter_number)s - %(chapter)s/%(playlist_index)s - %(title)s.%(ext)s', '--min-sleep-interval', '3', '--max-sleep-interval', '7', 'https://udemy.com/course/typescript-the-complete-developers-guide/', '--playlist-start', '169'] [debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252 [debug] youtube-dl version 2019.09.28 [debug] Python version 3.4.4 (CPython) - Windows-10-10.0.17763 [debug] exe versions: none [debug] Proxy map: {} [udemy:course] Downloading login popup [udemy:course] Logging in [udemy:course] course: Downloading webpage [udemy:course] 2337318: Enrolling in the course [udemy:course] 2337318: Downloading course curriculum [download] Downloading playlist: 2337318 [udemy:course] playlist 2337318: Collected 273 video ids (downloading 105 of them) [download] Downloading video 1 of 105 [udemy] Downloading login popup [udemy] 15067000: Downloading webpage [udemy] 15067000: Downloading lecture JSON [udemy] 15067000: Downloading lecture JSON ERROR: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpyi91grvc\build\youtube_dl\extractor\common.py", line 627, in _request_webpage File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpyi91grvc\build\youtube_dl\YoutubeDL.py", line 2237, in urlopen File "C:\Python\Python34\lib\urllib\request.py", line 470, in open File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response File "C:\Python\Python34\lib\urllib\request.py", line 508, in error File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default`

Have been getting this will all Udemy courses. Tried with both "-u -p" as well as the "--cookies" flags. Both give the same error as below.

[debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: [u'--cookies', u'/home/Downloads/udemy-cookies.txt', u'-o', u'~/Videos/%(playlist)s/%(chapter_number)s - %(chapter)s/%(title)s.%(ext)', u'https://www.udemy.com/course/elixir-phoenix-real-world-functional-programming/learn/lecture', u'--verbose', u'-F'] [debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8 [debug] youtube-dl version 2019.09.28 [debug] Python version 2.7.15+ (CPython) - Linux-5.0.0-31-generic-x86_64-with-Ubuntu-18.04-bionic [debug] exe versions: ffmpeg 3.4.6, ffprobe 3.4.6, phantomjs 2.1.1, rtmpdump 2.4 [debug] Proxy map: {} [udemy:course] course: Downloading webpage [udemy:course] 1976564: Downloading course curriculum [download] Downloading playlist: 1976564 [udemy:course] playlist 1976564: Collected 41 video ids (downloading 41 of them) [download] Downloading video 1 of 41 [udemy] 12391620: Downloading webpage [udemy] 12391620: Downloading lecture JSON [udemy] 12391620: Downloading lecture JSON ERROR: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. File "/usr/bin/youtube-dl/youtube_dl/extractor/common.py", line 627, in _request_webpage return self._downloader.urlopen(url_or_request) File "/usr/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2237, in urlopen return self._opener.open(req, timeout=self._socket_timeout) File "/usr/lib/python2.7/urllib2.py", line 435, in open response = meth(req, response) File "/usr/lib/python2.7/urllib2.py", line 548, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python2.7/urllib2.py", line 473, in error return self._call_chain(args) File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain result = func(args) File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)

I got the same error and I found out if I remove /course/ in the url, then it works. For ex, instead of https://www.udemy.com/course/linux-bash-shell-scripting-complete-guide-incl-awk-sed/, you should use https://www.udemy.com/linux-bash-shell-scripting-complete-guide-incl-awk-sed/. Hope this helps.

I got the same issue with Linkedin Learning. I'm using cookies instead of user & password. I just update cookies files ( with newer contents ) and :

IT'S WORKS

What do I use ?

cookie.txt ( extension available for chrome, and else )
youtube-dl :smile_cat:

Command

youtube-dl --cookies "path\to\cookies.txt" -o "path\to\%(playlist_title)s\%(chapter)s\%(playlist_index)s - %(title)s.%(ext)s" --playlist-start 1 --write-sub --sub-lang fr --write-description --write-info-json --write-annotations URL_PLAYLIST_HERE

URL_PLAYLIST_HERE : your playlist url

Feel free to adapt to your needed.

I've tried out the solution recommended by tuanbs and can confirm this work

I got the same error and I found out if I remove /course/ in the url, then it works. For ex, instead of https://www.udemy.com/course/linux-bash-shell-scripting-complete-guide-incl-awk-sed/, you should use https://www.udemy.com/linux-bash-shell-scripting-complete-guide-incl-awk-sed/. Hope this helps.

My results are shown below for reference.

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', '-u', 'PRIVATE', '-p', 'PRIVATE', '-F', '-o', '%(playlist)s/%(chapter_number)s - %(chapter)s/%(playlist_index)s. %(title)s.%(ext)s
', 'https://www.udemy.com/linux-bash-shell-scripting-complete-guide-incl-awk-sed/']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2019.09.28
[debug] Python version 3.4.4 (CPython) - Windows-7-6.1.7601-SP1
[debug] exe versions: none
[debug] Proxy map: {}
[udemy:course] Downloading login popup
[udemy:course] Logging in
[udemy:course] linux-bash-shell-scripting-complete-guide-incl-awk-sed: Downloading webpage
[udemy:course] 969354: Downloading course curriculum
[download] Downloading playlist: 969354
[udemy:course] playlist 969354: Collected 94 video ids (downloading 94 of them)
[download] Downloading video 1 of 94
[udemy] Downloading login popup
[udemy] 14300420: Downloading webpage
[udemy] 14300420: Downloading lecture JSON
[udemy] 17493458: Downloading m3u8 information
[info] Available formats for 17493458:
format code  extension  resolution note
144p         mp4        256x144     122k , libx264@  60k, 30fps, aac  @ 62k (44100Hz)
hls-170      mp4        256x144     170k , avc1.4d400c, 30.0fps, mp4a.40.2
360p         mp4        640x360     318k , libx264@ 193k, 30fps, aac  @125k (44100Hz)
hls-390      mp4        640x360     390k , avc1.4d401e, 30.0fps, mp4a.40.2
480p         mp4        854x480     437k , libx264@ 312k, 30fps, aac  @125k (44100Hz)
hls-519      mp4        854x480     519k , avc1.4d401f, 30.0fps, mp4a.40.2
720p         mp4        1280x720    704k , libx264@ 579k, 30fps, aac  @125k (44100Hz)
hls-812      mp4        1280x720    812k , avc1.4d401f, 30.0fps, mp4a.40.2
hls-1116     mp4        1920x1080  1116k , avc1.4d4028, 30.0fps, mp4a.40.2 (best)
[download] Downloading video 2 of 94
[udemy] 14203212: Downloading webpage
[udemy] 14203212: Downloading lecture JSON
[udemy] 17367590: Downloading m3u8 information
[info] Available formats for 17367590:
format code  extension  resolution note
144p         mp4        256x144     124k , libx264@  62k, 30fps, aac  @ 62k (44100Hz)
hls-172      mp4        256x144     172k , avc1.4d400c, 30.0fps, mp4a.40.2
360p         mp4        640x360     314k , libx264@ 189k, 30fps, aac  @125k (44100Hz)
hls-385      mp4        640x360     385k , avc1.4d401e, 30.0fps, mp4a.40.2
480p         mp4        480p
hls-505      mp4        854x480     505k , avc1.4d401f, 30.0fps, mp4a.40.2
720p         mp4        1280x720    678k , libx264@ 553k, 30fps, aac  @125k (44100Hz)
hls-779      mp4        1280x720    779k , avc1.4d401f, 30.0fps, mp4a.40.2
hls-1104     mp4        1920x1080  1104k , avc1.4d4028, 30.0fps, mp4a.40.2 (best)
[download] Downloading video 3 of 94
[udemy] 14203228: Downloading webpage

ERROR: Interrupted by user

I've not tried out ludndev's proposed solution, but I believe it can work. As such, I appreciate all the support and help provided in helping to resolve this issue. I'll be closing this issue in 2 days time so as to give enough time to other users try out and comment on the solutions suggested by the above mentioned users.

Thank you all for the support and assistance in helping to resolve this issue. As, I have tested the solution suggested by tuanbs successfully in my earlier post, I'm now closing this issue.

Continued in #30802.

ytdl-org / youtube-dl