search

coursera-dl / coursera-dl

Script for downloading Coursera.org videos and naming them.
GNU Lesser General Public License v3.0
9.35k stars 2.21k forks source link

403 client error, Forbidden for URL #744

Open zenny opened 4 years ago

zenny commented 4 years ago

Hi,

Got the same problem as in https://github.com/coursera-dl/coursera-dl/issues/723, even with the solution provided by @sc268 at https://github.com/coursera-dl/coursera-dl/issues/702#issuecomment-568228372 and @BadWindShield at https://github.com/coursera-dl/coursera-dl/issues/702#issuecomment-568918178.

The output error looks like the following>

/shared/projects/my-coursera/local/lib/python3.4/site-packages/requests/__init__.py:80: RequestsDependencyWarning: urllib3 (1.25.7) or chardet (3.0.4) doesn't match a supported version!
  RequestsDependencyWarning)
coursera_dl version 0.11.5
Downloading class: contemporary-art (1 / 1)
Parsing syllabus of on-demand course (id=eIuy3sp7Eem_qg46ES4SQA). This may take some time, please be patient ...
Error 403 Client Error: Forbidden for url: https://api.coursera.org/api/memberships.v1?includes=courseId,courses.v1&q=me&showHidden=true&filter=current,preEnrolled getting page https://api.coursera.org/api/memberships.v1?includes=courseId,courses.v1&q=me&showHidden=true&filter=current,preEnrolled
The server replied: {"errorCode":"Not Authorized","message":null,"details":null}
HTTPError 403 Client Error: Forbidden for url: https://api.coursera.org/api/memberships.v1?includes=courseId,courses.v1&q=me&showHidden=true&filter=current,preEnrolled

Tried many a times if it works arbitrarily as @aspiringguru mentioned at https://github.com/coursera-dl/coursera-dl/issues/723#issuecomment-524112388, but no go in my case!

Any input?

Cheers,

leijianbin commented 4 years ago
  1. Login the coursera with Chrome, and use the export the ccokies.txt chrome plugin to export the cookies.txt info
  2. Create a new folder you want to store the download files, and create a new file called: cookies.txt with the cookies.txt info from step 1
  3. Run the coursera-dl -u -p course-id in the same folder, it should works
numlockkey commented 4 years ago

I'm experiencing the same problem.

pykong commented 4 years ago

@leijianbin Tried this many times: Did not work. This is a known problem - discussed in several issues. As of now coursera-dl appears to be no longer working.

zenny commented 4 years ago
  1. Login the coursera with Chrome, and use the export the ccokies.txt chrome plugin to export the cookies.txt info
  2. Create a new folder you want to store the download files, and create a new file called: cookies.txt with the cookies.txt info from step 1
  3. Run the coursera-dl -u -p course-id in the same folder, it should works

@leijianbin Thanks. Tried and tested as stated in OP, but no go! :(

eduardosteps commented 4 years ago

Cannot login on coursera.org: 400 Client Error: Bad Request for url: https://api.coursera.org/api/login/v3

wbchn commented 4 years ago

I got the same error when download course, and tried many times, after remove --cauth, it works. but still need --cauth when list courses(#702).

btw, using: coursera_dl version 0.11.5

pykong commented 4 years ago

@wbchn Could you share the exact steps and commands you performed to conduct a successful download?

wbchn commented 4 years ago

@wbchn Could you share the exact steps and commands you performed to conduct a successful download?

Download

  1. make sure remove --cauth param in coursera-dl.conf if used.
  2. run download command, here my example: coursera-dl -u 'username' -p 'password' --hook "$PWD/process.sh" ${course}.

List

Test on my pc yesterday:

Operator with --cauth without --cauth
List fine #702 sometimes
Download 403 Error fine

More information:

pykong commented 4 years ago

@wbchn Thanks!

pykong commented 4 years ago

I is the first time I got coursera-dl working. I have used the most recent release (from Dec 19) and specified --cauth as given by the cookies.txt Chome extension.

I have not tested whether I need to be logged into Coursera with my browser or not while downloading.

I have not tried out the list courses functionality.

Harshitbasti commented 4 years ago

When I run user@localhost:~$ /home/user/.local/bin/coursera-dl -u username -p password course name I found following error. How to fix it?

coursera_dl version 0.11.5 Traceback (most recent call last): File "/home/user/.local/bin/coursera-dl", line 8, in sys.exit(main()) File "/home/user/.local/lib/python2.7/site-packages/coursera/coursera_dl.py", line 239, in main login(session, args.username, args.password) File "/home/user/.local/lib/python2.7/site-packages/coursera/cookies.py", line 155, in login raise AuthenticationFailed('Cannot login on coursera.org: %s' % e) coursera.cookies.AuthenticationFailed: Cannot login on coursera.org: 400 Client Error: Bad Request for url: https://api.coursera.org/api/login/v3

zenny commented 4 years ago

@wbchn Could you share the exact steps and commands you performed to conduct a successful download?

Download

1. make sure remove `--cauth` param in `coursera-dl.conf` if used.

2. run download command, here my example: `coursera-dl -u 'username'  -p 'password' --hook "$PWD/process.sh" ${course}`.

List

* method a: `coursera-dl -u 'username'  -p 'password' --cauth 'cauth-string' --list-courses`

* method b: put `--cauth 'cauth-string'` in `coursera-dl.conf`, and run `coursera-dl -u 'username'  -p 'password' --list-courses`

Test on my pc yesterday: Operator with --cauth without --cauth List fine #702 sometimes Download 403 Error fine

More information:

* system: Debian + Python 3.7.4

* for some reason, I used a socks proxy on google cloud vm to improve my network.

This does not seem to work even all flags are in the command line as @wbchn advised:

/shared/projects/my-coursera/local/lib/python3.4/site-packages/requests/__init__.py:80: RequestsDependencyWarning: urllib3 (1.25.7) or chardet (3.0.4) doesn't match a supported version!
  RequestsDependencyWarning)
coursera_dl version 0.11.5
Listing enrolled courses
Traceback (most recent call last):
  File "/shared/projects/my-coursera/coursera/coursera/cookies.py", line 148, in login
    r.raise_for_status()
  File "/shared/projects/my-coursera/local/lib/python3.4/site-packages/requests/models.py", line 935, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://api.coursera.org/api/login/v3

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./coursera-dl", line 6, in <module>
    coursera_dl.main()
  File "/shared/projects/my-coursera/coursera/coursera/coursera_dl.py", line 232, in main
    list_courses(args)
  File "/shared/projects/my-coursera/coursera/coursera/coursera_dl.py", line 108, in list_courses
    login(session, args.username, args.password)
  File "/shared/projects/my-coursera/coursera/coursera/cookies.py", line 155, in login
    raise AuthenticationFailed('Cannot login on coursera.org: %s' % e)
coursera.cookies.AuthenticationFailed: Cannot login on coursera.org: 400 Client Error: Bad Request for url: https://api.coursera.org/api/login/v3

Appears as if coursera has changed something upstream, I guess. FYI, coursera-dl is already up-to-date.

Cheers and stay safe, /z

wbchn commented 4 years ago

@zenny , can you share your full command line(with coursera-dl.conf) with error output?

I think the error you met now is #702 , not 403 client error now: coursera.cookies.AuthenticationFailed: Cannot login on coursera.org: 400 Client Error: Bad Request for url: https://api.coursera.org/api/login/v3

zenny commented 4 years ago

@wbchn please check the output here: https://github.com/coursera-dl/coursera-dl/issues/702#issuecomment-614715274 (voidlinux)

The above 403 is in debian jessie on a different machine, to make things clear.

The command I was successful in the past are:

./coursera-dl -u <username@name.tld> -p <XYZYODLKA> -cauth <STRINGfromCOOKIES.txt> --list-courses

Thanks and stay safe!

eduardosteps commented 4 years ago

Hi,

Got the same problem as in #723, even with the solution provided by @sc268 at #702 (comment) and @BadWindshield at #702 (comment).

The output error looks like the following>

/shared/projects/my-coursera/local/lib/python3.4/site-packages/requests/__init__.py:80: RequestsDependencyWarning: urllib3 (1.25.7) or chardet (3.0.4) doesn't match a supported version!
  RequestsDependencyWarning)
coursera_dl version 0.11.5
Downloading class: contemporary-art (1 / 1)
Parsing syllabus of on-demand course (id=eIuy3sp7Eem_qg46ES4SQA). This may take some time, please be patient ...
Error 403 Client Error: Forbidden for url: https://api.coursera.org/api/memberships.v1?includes=courseId,courses.v1&q=me&showHidden=true&filter=current,preEnrolled getting page https://api.coursera.org/api/memberships.v1?includes=courseId,courses.v1&q=me&showHidden=true&filter=current,preEnrolled
The server replied: {"errorCode":"Not Authorized","message":null,"details":null}
HTTPError 403 Client Error: Forbidden for url: https://api.coursera.org/api/memberships.v1?includes=courseId,courses.v1&q=me&showHidden=true&filter=current,preEnrolled

Tried many a times if it works arbitrarily as @aspiringguru mentioned at #723 (comment), but no go in my case!

Any input?

Cheers,

I found the solution, guys!

First of all, check the youtube-dl version and upgrade if necessary ;)

Make sure to have the packages coursera-dl and edx-dl installed

if not, run: pip install coursera-dl edx-dl

Download tthis Chrome Extension: https://chrome.google.com/webstore/detail/coursera-authentication-h/fkbgecmligfnknoclmgaebbcoodcacnf

Create a file coursera-dl.conf in the same directory of the coursera-dl with the text below: --username --password --subtitle-language en --cauth <paste_your_cookie_here_from_the_chrome_extensionnamedCoursera Authentication Helper> --download-quizzes --download-notebooks

Enroll the courses you want to download

for edx run: edx-dl -u -p --list-courses edx-dl -u -p

for coursera run: coursera-dl -u -p --list-courses coursera-dl

IMPORTANT BUG NOTE!

COURSERA-DL is not working at first. It's necessary to:

1) Logout from the account and close all tabs;

2) Enter the website 'https://learner.coursera.help/hc/en-us' and login from this page;

3) Run the script. It'll raise an error, but try again and again the same commmand until it work (it's working on fifth or sixth attempt, sometimes more)

zenny commented 4 years ago

Hi, Got the same problem as in #723, even with the solution provided by @sc268 at #702 (comment) and @BadWindshield at #702 (comment). The output error looks like the following>

/shared/projects/my-coursera/local/lib/python3.4/site-packages/requests/__init__.py:80: RequestsDependencyWarning: urllib3 (1.25.7) or chardet (3.0.4) doesn't match a supported version!
  RequestsDependencyWarning)
coursera_dl version 0.11.5
Downloading class: contemporary-art (1 / 1)
Parsing syllabus of on-demand course (id=eIuy3sp7Eem_qg46ES4SQA). This may take some time, please be patient ...
Error 403 Client Error: Forbidden for url: https://api.coursera.org/api/memberships.v1?includes=courseId,courses.v1&q=me&showHidden=true&filter=current,preEnrolled getting page https://api.coursera.org/api/memberships.v1?includes=courseId,courses.v1&q=me&showHidden=true&filter=current,preEnrolled
The server replied: {"errorCode":"Not Authorized","message":null,"details":null}
HTTPError 403 Client Error: Forbidden for url: https://api.coursera.org/api/memberships.v1?includes=courseId,courses.v1&q=me&showHidden=true&filter=current,preEnrolled

Tried many a times if it works arbitrarily as @aspiringguru mentioned at #723 (comment), but no go in my case! Any input? Cheers,

I found the solution, guys!

First of all, check the youtube-dl version and upgrade if necessary ;)

Make sure to have the packages coursera-dl and edx-dl installed

if not, run: pip install coursera-dl edx-dl

Download tthis Chrome Extension: https://chrome.google.com/webstore/detail/coursera-authentication-h/fkbgecmligfnknoclmgaebbcoodcacnf

Create a file coursera-dl.conf in the same directory of the coursera-dl with the text below: --username --password --subtitle-language en --cauth <paste_your_cookie_here_from_the_chrome_extensionnamedCoursera Authentication Helper> --download-quizzes --download-notebooks

Enroll the courses you want to download

for edx run: edx-dl -u -p --list-courses edx-dl -u -p

for coursera run: coursera-dl -u -p --list-courses coursera-dl

IMPORTANT BUG NOTE!

COURSERA-DL is not working at first. It's necessary to:

1. Logout from the account and close all tabs;

2. Enter the website 'https://learner.coursera.help/hc/en-us' and login from this page;

3. Run the script. It'll raise an error, but try again and again the same commmand until it work (it's working on fifth or sixth attempt, sometimes more)

Did everything as above including logged out from the account, closed the tabs, entered the login from help pages tried several times, still getting 400 error! :(

arnoldmw commented 4 years ago

@eduardosteps thank you. It worked though at the end I typed 'coursera-dl.exe -u ' after I followed all the other steps.

marwa-dosoky commented 4 years ago

the chrome chrome extension has been removed. any suggestions?

marwa-dosoky commented 4 years ago

the chrome chrome extension has been removed. any suggestions?

never mind, exposed the cookie and edited the --cauth

eduardosteps commented 4 years ago

the chrome chrome extension has been removed. any suggestions?

It's available here on Github as well: https://github.com/e-learning-archive/browser-extension

mararkeri commented 4 years ago

Create a file coursera-dl.conf in the same directory of the coursera-dl with the text below: --username --password --subtitle-language en --cauth

cauth value will be available only if you login to coursera and then check in cookies. You may check the following video to debug the error: https://youtu.be/xpy0H4T9zHU

If you don't know how to use coursera-dl, you can check the following video: https://youtu.be/wP9ZyTRIAS4

eduardosteps commented 4 years ago

Create a file coursera-dl.conf in the same directory of the coursera-dl with the text below: --username --password --subtitle-language en --cauth

cauth value will be available only if you login to coursera and then check in cookies. You may check the following video: https://youtu.be/xpy0H4T9zHU

AWESOME!!!! Thanks a lot!

Manukapp commented 3 years ago

I've been going through a lot of errors, with unsuccessful trials, but so far I found some solutions inspired by @zenny . BTW I didn't find the Chrome Extension, but yes to the Firefox extension:

https://stackoverflow.com/questions/68614376/coursera-dl-not-downloading-classes-command-prompt-windows/68615250#68615250

kikesainz commented 2 years ago

Create a file coursera-dl.conf in the same directory of the coursera-dl with the text below: --username --password --subtitle-language en --cauth

cauth value will be available only if you login to coursera and then check in cookies. You may check the following video to debug the error: https://youtu.be/xpy0H4T9zHU

If you don't know how to use coursera-dl, you can check the following video: https://youtu.be/wP9ZyTRIAS4

This video sorts the problem out. You don't need any extension to get the cookie, but if you want to use an extension to manage cookies try EditThisCookie. It's a great one.