Cannot download Udemy paid courses with constant 403 error to retrieve JSON metadata.

[edwardtian]

Checklist

• [x] I'm reporting a broken site support
• [x] I've verified that I'm running youtube-dl version 2021.12.17
• [x] I've checked that all provided URLs are alive and playable in a browser
• [x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
• [x] I've searched the bugtracker for similar issues including closed ones

Verbose log

./youtube-dl --cookies cookies.txt --user-agent 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHT
ML, like Gecko) Chrome/100.0.4896.127 Safari/537.36 Edg/100.0.1185.50' --verbose -a urls.txt                                               
[debug] System config: []                                                                                                                  
[debug] User config: []                                                                                                                    
[debug] Custom config: []                                                                                                                  
[debug] Command-line args: ['--cookies', 'cookies.txt', '--user-agent', 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTM
L, like Gecko) Chrome/100.0.4896.127 Safari/537.36 Edg/100.0.1185.50', '--verbose', '-a', 'urls.txt']                                      
[debug] Batch file urls: ['https://www.udemy.com/course/unreal-engine-the-ultimate-game-developer-course/learn/', 'https://www.udemy.com/co
urse/unreal-engine-course/learn/', 'https://www.udemy.com/course/unreal-engine-5-cpp-multiplayer-shooter/learn/', 'https://www.udemy.com/co
urse/unrealcourse/learn/', 'https://www.udemy.com/course/blenderlowpoly/learn/', 'https://www.udemy.com/course/blender-low-poly-characters/
learn/', 'https://www.udemy.com/course/blendertutorial/learn/', 'https://www.udemy.com/course/learn-how-to-do-cloth-animation-in-unreal-eng
ine-4/learn/', 'https://www.udemy.com/course/how-to-create-a-movie-in-unreal-engine-5-beginners-edition/learn/', 'https://www.udemy.com/cou
rse/unrealvr/learn/', 'https://www.udemy.com/course/ue5-steam/learn/', 'https://www.udemy.com/course/remaking-rocket-league-in-unreal-engin
e-5-pocket-league/learn/', 'https://www.udemy.com/course/blender-quickstart/learn/', 'https://www.udemy.com/course/blender-low-poly-landsca
pes/learn/', 'https://www.udemy.com/course/pro-unreal-engine-game-coding/learn/', 'https://www.udemy.com/course/unreal-engine-fx/learn/', '
https://www.udemy.com/course/learn-blender-28/learn/', 'https://www.udemy.com/course/unreal-engine-4-create-an-arcade-classic/learn/', 'htt
ps://www.udemy.com/course/unreal-engine-5-blueprints-fps/learn/', 'https://www.udemy.com/course/blender-low-poly-course/learn/']           
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8                                                                           
[debug] youtube-dl version 2021.12.17                                                                                                      
[debug] Python version 3.10.4 (CPython) - Linux-4.4.180+-x86_64-with-glibc2.35                                                             
[debug] exe versions: ffmpeg 4.4.1, ffprobe 4.4.1                                                                                          
[debug] Proxy map: {}                                                                                                                      
[udemy:course] course: Downloading webpage                                                                                                 
[udemy:course] 2293851: Downloading course curriculum                                                                                      
[download] Downloading playlist: 2293851                                                                                                   
[udemy:course] playlist 2293851: Collected 100 video ids (downloading 100 of them)                                                         
[download] Downloading video 1 of 100                                                                                                      
[udemy] 14173595: Downloading webpage                                                                                                      
[udemy] 14173595: Downloading lecture JSON                                                                                                 

[udemy] 14173595: Downloading lecture JSON                                                                                                 
ERROR: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on ht
tps://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --ver
bose flag and include its complete output.                                                                                                 
  File "/download/./youtube-dl/youtube_dl/extractor/common.py", line 634, in _request_webpage                                              
    return self._downloader.urlopen(url_or_request)                                                                                        
  File "/download/./youtube-dl/youtube_dl/YoutubeDL.py", line 2288, in urlopen                                                             
    return self._opener.open(req, timeout=self._socket_timeout)                                                                            
  File "/usr/lib/python3.10/urllib/request.py", line 525, in open                                                                          
    response = meth(req, response)                                                                                                         
  File "/usr/lib/python3.10/urllib/request.py", line 634, in http_response                                                                 
    response = self.parent.error(                                                                                                          
  File "/usr/lib/python3.10/urllib/request.py", line 563, in error                                                                         
    return self._call_chain(*args)                                                                                                         
  File "/usr/lib/python3.10/urllib/request.py", line 496, in _call_chain                                                                   
    result = func(*args)                                                                                                                   
  File "/usr/lib/python3.10/urllib/request.py", line 643, in http_error_default                                                            
    raise HTTPError(req.full_url, code, msg, hdrs, fp) 

Description

youtube-dl works smoothly with youtube, but for paid Udemy video, it won't work: Here is a list of URL I've tried: https://www.udemy.com/course/unreal-engine-the-ultimate-game-developer-course/learn/
https://www.udemy.com/course/unreal-engine-course/learn/
https://www.udemy.com/course/unreal-engine-5-cpp-multiplayer-shooter/learn/
https://www.udemy.com/course/unrealcourse/learn/
https://www.udemy.com/course/blenderlowpoly/learn/

I put these URLs into a file, and use -a to tell youtube-dl to read from file. According to the suggestion of waiting a good amount of time in other guys' closed issues, I have been trying twice a day during the past week, with no luck, it always breaks at retrieving JSON metadata with HTTP403 Error.

Thanks very much for your great work! Hope it can be fixed.

Edward

[dirkf]

Duplicate of #22306, closed as solved, and, more immediately, of #30802.