search

ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
131.89k stars 10k forks source link

Udemy.com HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>) #22705

Closed vikiaiteam closed 2 years ago

vikiaiteam commented 5 years ago

Checklist

Verbose log


 C:\udemy-downloaded>youtube-dl -v -u xxx@gmail.com -p xxx --cookies C:\udemy-downloaded\udemy-cookies.txt -o "C:\udemy-downloaded\%(
playlist)s/%(chapter_number)s-%(chapter)s/%(playlist_index)s.%(title)s.%(ext)s" https://www.udemy.com/course/academic-ielts-writing-task-1 
s
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '-u', 'PRIVATE', '-p', 'PRIVATE', '--cookies', 'C:\\udemy-downloaded\\udemy-cookies.txt', '-o', 'C:\\udemy-downloaded\\%(pl
aylist)s/%(chapter_number)s-%(chapter)s/%(playlist_index)s.%(title)s.%(ext)s', 'https://www.udemy.com/course/academic-ielts-writing-task-1', '--convert-subs', 'vtt']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252
[debug] youtube-dl version 2019.09.28
[debug] Python version 3.7.4 (CPython) - Windows-7-6.1.7601-SP1
[debug] exe versions: ffmpeg N-93640-ga087764847, ffprobe N-93640-ga087764847
[debug] Proxy map: {}
[udemy:course] Downloading login popup
[udemy:course] course: Downloading webpage
[udemy:course] 2205064: Downloading course curriculum
[download] Downloading playlist: 2205064
[udemy:course] playlist 2205064: Collected 43 video ids (downloading 43 of them)
[download] Downloading video 1 of 43
[udemy] Downloading login popup
[udemy] 13580764: Downloading webpage
[udemy] 13580764: Downloading lecture JSON
[udemy] 13580764: Downloading lecture JSON
ERROR: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://yt-dl.org/bug . Make sure you ar
e using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "c:\python37\lib\site-packages\youtube_dl\extractor\common.py", line 627, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "c:\python37\lib\site-packages\youtube_dl\YoutubeDL.py", line 2237, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "c:\python37\lib\urllib\request.py", line 531, in open
    response = meth(req, response)
  File "c:\python37\lib\urllib\request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "c:\python37\lib\urllib\request.py", line 569, in error
    return self._call_chain(*args)
  File "c:\python37\lib\urllib\request.py", line 503, in _call_chain
    result = func(*args)
  File "c:\python37\lib\urllib\request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)```

## Description

<!--
Provide an explanation of your issue in an arbitrary form. Provide any additional information, suggested solution and as much context and examples as possible.
If work on your issue requires account credentials please provide them or explain how one can obtain them.
-->

WRITE DESCRIPTION HERE
luizconrado commented 5 years ago

I have the exact same problem. Do we have any news about this bug?

jeaye commented 4 years ago

This has been reported many times, but I haven't yet seen any info on it. I've tried with a cookies.txt file and also with manually specifying -u and -p. The cookies file gets further, as shown in the logs above, but only as far as seeing all of the videos to download. Once it tries downloading the first video, the 403 shows up again.

Shineson1001 commented 4 years ago

I have the same problem for many days. In the Meanwhile i use udemy-dl (in a docker container). It works fine.

mabdelghaniML commented 4 years ago

I experienced the same issue from Git-bash and from the windows version. Any help or updates?

Darkcast commented 4 years ago

I have the same problem for many days. In the Meanwhile i use udemy-dl (in a docker container). It works fine.

That tool is also broken it doesnt download the videos either

Shineson1001 commented 4 years ago

I have the same problem for many days. In the Meanwhile i use udemy-dl (in a docker container). It works fine.

That tool is also broken it doesnt download the videos either

it works

root@nuc1:/opt/docker-udemydl # cat Dockerfile FROM python:alpine

RUN apk add --no-cache --update gcc libc-dev libffi-dev openssl-dev git && \ git clone https://github.com/r0oth3x49/udemy-dl.git && \ cd udemy-dl && \ pip install -r requirements.txt && \ mkdir -p /course

VOLUME ["/course"]

ENTRYPOINT ["python", "/udemy-dl/udemy-dl.py", "-o", "/course"]

docker build -t xxx/your-udemy-dl-image:1.0 . --no-cache docker run -it --rm -v $(pwd):/course --name tempudemy-dl xxx/your-udemy-dl-image:1.0 -u your-udemy-user -p your-udemy-password https://www.udemy.com/course/xxx

davidhenley commented 4 years ago

@Darkcast it works just fine with the instructions in the repo

MPW1412 commented 4 years ago

403 looks like a missmatch in the Metadata. Which data does the udemy-dl script use?

pavelloz commented 4 years ago

I tried to use gist instructions described here: https://gist.github.com/barbietunnie/8531d9c26cd1c0668e7278c7c4ba5853

But without success.

Heres what i got on current youtube-dl:

Downloads|⇒ youtube-dl --cookies cookie.txt https://www.udemy.com/course/hacking-real-websites-legally-2 --verbose
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--cookies', u'cookie.txt', u'https://www.udemy.com/course/hacking-real-websites-legally-2', u'--verbose']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2020.01.24
[debug] Python version 2.7.17 (CPython) - Darwin-18.7.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 4.2.2, ffprobe 4.2.2, rtmpdump 2.4
[debug] Proxy map: {}
[udemy:course] course: Downloading webpage
[udemy:course] 2664834: Downloading course curriculum
[download] Downloading playlist: 2664834
[udemy:course] playlist 2664834: Collected 28 video ids (downloading 28 of them)
[download] Downloading video 1 of 28
[udemy] 17022386: Downloading webpage
[udemy] 17022386: Downloading lecture JSON
[udemy] 17022386: Downloading lecture JSON
ERROR: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 627, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2237, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/local/Cellar/python@2/2.7.17_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/usr/local/Cellar/python@2/2.7.17_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/local/Cellar/python@2/2.7.17_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 473, in error
    return self._call_chain(*args)
  File "/usr/local/Cellar/python@2/2.7.17_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/usr/local/Cellar/python@2/2.7.17_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 556, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
pavelloz commented 4 years ago

@Shineson1001

For some reason my docker doesnt like the line breaks in this dockerfile:

Sending build context to Docker daemon  2.048kB
Error response from daemon: Dockerfile parse error line 4: unknown instruction: GIT

Splitting them into multiple RUNs / joining them to one line solved the issue. Thanks for your input.

This is how i solved it:

FROM python:alpine

RUN apk add --no-cache --update gcc libc-dev libffi-dev openssl-dev git

RUN git clone https://github.com/r0oth3x49/udemy-dl.git && cd udemy-dl && pip install -r requirements.txt && mkdir -p /course

VOLUME ["/course"]

ENTRYPOINT ["python", "/udemy-dl/udemy-dl.py", "-o", "/course"]
jose1711 commented 4 years ago

Same here, it dies in exactly same spot everytime I run it:

..
[download] Destination: 30318/3.03. Javascript Essentials/14.0303 Boolean Types And Comparison.mp4
[download] 100% of 3.05MiB in 00:00
[download] Downloading video 15 of 92
[udemy] 170627: Downloading webpage
ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
petr-hybler commented 4 years ago

Hello guys same here ... any update? It would be helpful to fix it so we can watch those courses offline

OS: Linux

youtube-dl -u #myUsername# -p #MyPassword# https://udemy.com/course/the-ultimate-drawing-course-beginner-to-advanced/ --verbose [debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: [u'-u', u'PRIVATE', u'-p', u'PRIVATE', u'https://udemy.com/course/the-ultimate-drawing-course-beginner-to-advanced/', u'--verbose'] [debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8 [debug] youtube-dl version 2020.03.24 [debug] Python version 2.7.17 (CPython) - Linux-5.3.0-45-generic-x86_64-with-Ubuntu-18.04-bionic [debug] exe versions: ffmpeg 3.4.6, ffprobe 3.4.6, phantomjs 2.1.1, rtmpdump 2.4 [debug] Proxy map: {} [udemy:course] Downloading login popup [udemy:course] Logging in [udemy:course] course: Downloading webpage [udemy:course] 874012: Enrolling in the course [udemy:course] 874012: Downloading course curriculum ERROR: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. File "/usr/local/lib/python2.7/dist-packages/youtube_dl/extractor/common.py", line 627, in _request_webpage return self._downloader.urlopen(url_or_request) File "/usr/local/lib/python2.7/dist-packages/youtube_dl/YoutubeDL.py", line 2238, in urlopen return self._opener.open(req, timeout=self._socket_timeout) File "/usr/lib/python2.7/urllib2.py", line 435, in open response = meth(req, response) File "/usr/lib/python2.7/urllib2.py", line 548, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python2.7/urllib2.py", line 473, in error return self._call_chain(*args) File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain result = func(*args) File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)

sebastianhritcu commented 4 years ago

Hi all,

The download will not work, because the course is not downloadable from the browser. You can check the api calls and search for "download_urls:" and you will see the value is set to null.

Maybe checking for the parameter from the api calls for "stream_urls:" and than try to download the video file.

api call example: { "stream_urls": { "Video": [ { "type":"video/mp4", "label":"720", "file":"https://bla bla bla with no extension" } ] } }

darkRaspberry commented 4 years ago

I also have this error

jadence commented 4 years ago

I was having the same 403 error despite purchasing the course. To fix it I did two things: 1) Clear my udemy cookies, log back into udemy.com, and then export a new cookies.txt 2) Changed the course URL from https://www.udemy.com/course/<course_name>/ to https://www.udemy.com/<course_name>/

After the above two steps running youtube-dl --cookies cookies.txt https://www.udemy.com/<course_name>/ --verbose finally worked!

petr-hybler commented 4 years ago

sorry ... even that does not work :(

Shineson1001 commented 4 years ago

hi @jadence, thanks, it works.

nbensa commented 4 years ago
  1. Changed the course URL from https://www.udemy.com/course/<course_name>/ to https://www.udemy.com/<course_name>/

This fixed it for me. I downloaded all the videos for the course I bought.

Thanks @jadence

Anil86 commented 4 years ago

@jadence , @nbensa Can you share your full youtube-dl's command syntax?

nbensa commented 4 years ago

@Anil86 

youtube-dl --cookies [path-to-cookies-file] https://www.udemy.com/<course-name>/

HTH

Anil86 commented 4 years ago

I tried to generate cookie file using EditThisCookie chrome extension. But it shows following error:

'cookies.txt' does not look like a Netscape format cookies file.

@nbensa What tool are you using to export cookie file?

nbensa commented 4 years ago

I used this extension: https://chrome.google.com/webstore/detail/cookiestxt/njabckikapfpffapmjgojcnbfjonfjfg

jadence commented 4 years ago

@jadence , @nbensa Can you share your full youtube-dl's command syntax?

@Anil86 - My command I used is youtube-dl --cookies cookies.txt https://www.udemy.com/<course_name>/ --verbose notes: • The --verbose flag almost certainly isn't necessary • My cookies.txt was placed in the same directory I ran the above command in • My cookies.txt file was generated using https://addons.mozilla.org/en-US/firefox/addon/cookies-txt/

Good luck!

petr-hybler commented 4 years ago

guys what os and version of youtube-dl are you running ??

My system Ubuntu: 20.04 LTS Gnome: 3.36.2 youtube-dl: 2020-06-06

and your way simply does not work.. always returns 403 :(

plus can you list down your python version and packages installed? might be something related to that as well

petr-hybler commented 4 years ago

holly crap it works now .. yes ... I uninstalled and reinstalled python and pip and it works now ...

khanu commented 4 years ago

I was having the same 403 error despite purchasing the course. To fix it I did two things:

1. Clear my udemy cookies, log back into udemy.com, and then export a new `cookies.txt`

2. Changed the course URL from `https://www.udemy.com/course/<course_name>/` to `https://www.udemy.com/<course_name>/`

After the above two steps running youtube-dl --cookies cookies.txt https://www.udemy.com/<course_name>/ --verbose finally worked!

This 🥇

Xonshiz commented 3 years ago

This seems to be broken now. Have been trying all the mentioned work-arounds, no luck.

petr-hybler commented 3 years ago

This seems to be broken now. Have been trying all the mentioned work-arounds, no luck.

arent u experiencing the one with amazon aws s3 storage instead? they recently migrated a lot of projects to aws and there is no way around atm :(

Xonshiz commented 3 years ago

Could be that and I've been reading and saw that Udemy has introduced DRM (at least on their new content), so that could also be in the play. Not sure how much this info is on point...

petr-hybler commented 3 years ago

Could be that and I've been reading and saw that Udemy has introduced DRM (at least on their new content), so that could also be in the play. Not sure how much this info is on point...

well check your --verbose output.. if you will see 403 on aws/s3 -> welcome to the club :(

Xonshiz commented 3 years ago

I don't see any URL in my verbose output... It's failing on downloading login pop up itself...

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-u', 'PRIVATE', '-p', 'PRIVATE', '-F', '-i', 'https://www.udemy.com/ableton-live-11-course/', '--verbose']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.05.16
[debug] Python version 3.9.5 (CPython) - macOS-11.3.1-x86_64-i386-64bit
[debug] exe versions: ffmpeg 4.4, ffprobe 4.4, rtmpdump 2.4
[debug] Proxy map: {}
[udemy:course] Downloading login popup
ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/usr/local/Cellar/youtube-dl/2021.5.16/libexec/lib/python3.9/site-packages/youtube_dl/extractor/common.py", line 634, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/local/Cellar/youtube-dl/2021.5.16/libexec/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 2288, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/local/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 523, in open
    response = meth(req, response)
  File "/usr/local/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 632, in http_response
    response = self.parent.error(
  File "/usr/local/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 561, in error
    return self._call_chain(*args)
  File "/usr/local/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 494, in _call_chain
    result = func(*args)
  File "/usr/local/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 641, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)

EDIT: After providing --print-traffic, I see that it's trying to make a call to Cloudflare CDN report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct ... Could it be that Yt-dl isn't able to bypass CF protection?

petr-hybler commented 3 years ago

is your cookie correct? try to delete all cache in your browser.... log in to udemy again, use "export cookies" add on and export the cookie .... and try again with verbose pls

Xonshiz commented 3 years ago

Did all of that already. That was my first guess... did a clean wipe off cookies on my browser. I've been using another tool "Udeler".. that seems to be downloading at least some courses, but, gets stuck whenever there's an attachment in the course.

ThomasChr commented 3 years ago

Here is a recent Issue for the AWS Problem: https://github.com/ytdl-org/youtube-dl/issues/28877 Update: On the other issue you can also have account credentials.

smaga38 commented 3 years ago

У меня была та же ошибка 403, несмотря на покупку курса. Чтобы исправить это, я сделал две вещи:

  1. Очистите мои файлы cookie udemy, снова войдите на udemy.com, а затем экспортируйте новый cookies.txt
  2. URL курса изменен с https://www.udemy.com/course/<course_name>/наhttps://www.udemy.com/<course_name>/

После двух вышеуказанных шагов запуск, youtube-dl --cookies cookies.txt https://www.udemy.com/<course_name>/ --verboseнаконец, сработал!

It worked for me too. Even without deleting cookies Thank you!!

dirkf commented 2 years ago

Continued in #22306, closed as solved.