search

coursera-dl / edx-dl

A simple tool to download video lectures from edx.org (and other openedx sites)
GNU Lesser General Public License v3.0
1.93k stars 640 forks source link

HTTP Error 403: Forbidden #637

Closed suvmpr closed 4 years ago

suvmpr commented 4 years ago

🚨Please review the Troubleshooting section before reporting any issue. Don't forget also to check the current issues to avoid duplicates.

Subject of the issue

Describe your issue here. Getting HTTPError: HTTP Error 403: Forbidden everytime, nothings works. Even on different OS'

Your environment

Steps to reproduce

Tell us how to reproduce this issue. Please provide us the course URL, and the specific subsection or unit if possible. https://courses.edx.org/courses/course-v1:WasedaX+JPC111x+1T2020/course/ https://courses.edx.org/courses/course-v1:LinuxFoundationX+LFS101x+1T2020/course/

Expected behaviour

Tell us what should happen. Downloads the course

Actual behaviour

Tell us what happens instead. If the script fails, please copy the entire output of the command or the stacktrace (don't forget to obfuscate your username and password). If you cannot copy the exception, attach a screenshot. edx_dl version 0.1.13 Password: Building initial headers for future requests. Getting initial CSRF token. Found CSRF token. Logging into Open edX site: https://courses.edx.org/login_ajax Extracting course information from dashboard. Traceback (most recent call last): File "/usr/local/bin/edx-dl", line 8, in sys.exit(main()) File "/usr/local/lib/python3.6/dist-packages/edx_dl/edx_dl.py", line 1023, in main for selected_course in selected_courses} File "/usr/local/lib/python3.6/dist-packages/edx_dl/edx_dl.py", line 1023, in for selected_course in selected_courses} File "/usr/local/lib/python3.6/dist-packages/edx_dl/edx_dl.py", line 184, in get_available_sections page = get_page_contents(url, headers) File "/usr/local/lib/python3.6/dist-packages/edx_dl/utils.py", line 58, in get_page_contents result = urlopen(Request(url, None, headers)) File "/usr/lib/python3.6/urllib/request.py", line 223, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.6/urllib/request.py", line 532, in open response = meth(req, response) File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.6/urllib/request.py", line 570, in error return self._call_chain(args) File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain result = func(args) File "/usr/lib/python3.6/urllib/request.py", line 650, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden

elnurgar commented 4 years ago

Author: THolding "I'm not an author of the tool, but you can fix it by changing line 425 of edx_dl.py which specifies the User-Agent attribute of the http request header. Change 'User-Agent': 'edX-downloader/0.01', to 'User-Agent': 'Mozilla/5.0', and it will work."

It worked for me.

suvmpr commented 4 years ago

Author: THolding "I'm not an author of the tool, but you can fix it by changing line 425 of edx_dl.py which specifies the User-Agent attribute of the http request header. Change 'User-Agent': 'edX-downloader/0.01', to 'User-Agent': 'Mozilla/5.0', and it will work."

It worked for me.

I tried changing that, but no luck ,i even changed it to my full user agent but same issue.

juanjrivasp commented 4 years ago

Author: THolding "I'm not an author of the tool, but you can fix it by changing line 425 of edx_dl.py which specifies the User-Agent attribute of the http request header. Change 'User-Agent': 'edX-downloader/0.01', to 'User-Agent': 'Mozilla/5.0', and it will work." It worked for me.

Worked for me too. Thanks!

suvmpr commented 4 years ago

Author: THolding "I'm not an author of the tool, but you can fix it by changing line 425 of edx_dl.py which specifies the User-Agent attribute of the http request header. Change 'User-Agent': 'edX-downloader/0.01', to 'User-Agent': 'Mozilla/5.0', and it will work."

It worked for me.

Thanks It worked now.

Learnpython-code commented 3 years ago

Hello everyone, I am new with python, Please help checking my results, I dont got any videos , only folders empty.

Result

C:\edx-dl-master>python edx-dl.py -u (username) https://courses.edx.org/courses/coursev1:URosarioX+URX01+1T2020/course/ edx_dl version 0.1.13 Password: Building initial headers for future requests. Getting initial CSRF token. Found CSRF token. Logging into Open edX site: https://courses.edx.org/login_ajax Extracting course information from dashboard. Downloading Diseño de sistemas de información gerencial para intranet con Micros oft Access [course-v1:URosarioX+URX01+1T2020/co] Downloading 5 section(s) Section 1: Generalidades Acerca del curso Section 2: Microsoft Access y Bases de Datos Relacionales Conceptos básicos Planear y crear una BDR Evaluación Section 3: Diseño de la interface - Consultas Visualizar información Modificar la BDR con consultas de acción Interacción con otros programas Evaluación Section 4: Diseño de la interface - Formularios y macros Ingresar datos a la BDR Panel de control personalizado Evaluación Section 5: Diseño de la interface - Informes Informes Evaluación Cierre Extracting all units information in parallel. Processing 'https://courses.edx.org/courses/course-v1:URosarioX+URX01+1T2020/jum p_to/block-v1:URosarioX+URX01+1T2020+type@sequential+block@ddbbb4394e4f4eeab5716 95c19842fc2' Processing 'https://courses.edx.org/courses/course-v1:URosarioX+URX01+1T2020/jum p_to/block-v1:URosarioX+URX01+1T2020+type@sequential+block@edcc3663b92546ee9f37d 4868d05ba30' Processing 'https://courses.edx.org/courses/course-v1:URosarioX+URX01+1T2020/jum p_to/block-v1:URosarioX+URX01+1T2020+type@sequential+block@7a917180012346c8b7f1d e5837729bbd' Processing 'https://courses.edx.org/courses/course-v1:URosarioX+URX01+1T2020/jum p_to/block-v1:URosarioX+URX01+1T2020+type@sequential+block@fdb672aa18b0485aa6954 19f493a5fd0' Processing 'https://courses.edx.org/courses/course-v1:URosarioX+URX01+1T2020/jum p_to/block-v1:URosarioX+URX01+1T2020+type@sequential+block@5b34eb36e50a4db6a9c4c 53e719546cf' Processing 'https://courses.edx.org/courses/course-v1:URosarioX+URX01+1T2020/jum p_to/block-v1:URosarioX+URX01+1T2020+type@sequential+block@c78e301110b54cff8a850 0c784e16d09' Processing 'https://courses.edx.org/courses/course-v1:URosarioX+URX01+1T2020/jum p_to/block-v1:URosarioX+URX01+1T2020+type@sequential+block@fcd257068abb4f588805d b3a15e0ba06' Processing 'https://courses.edx.org/courses/course-v1:URosarioX+URX01+1T2020/jum p_to/block-v1:URosarioX+URX01+1T2020+type@sequential+block@9205182f4d2b46ec93fd6 ff22d752fa6' Processing 'https://courses.edx.org/courses/course-v1:URosarioX+URX01+1T2020/jum p_to/block-v1:URosarioX+URX01+1T2020+type@sequential+block@f9a2c97a613a40169a016 67bb6aca2be' Processing 'https://courses.edx.org/courses/course-v1:URosarioX+URX01+1T2020/jum p_to/block-v1:URosarioX+URX01+1T2020+type@sequential+block@30549607116847379bc57 b4419084652' Processing 'https://courses.edx.org/courses/course-v1:URosarioX+URX01+1T2020/jum p_to/block-v1:URosarioX+URX01+1T2020+type@sequential+block@09f8ee9e3295491495749 4d87da8a4bc' Processing 'https://courses.edx.org/courses/course-v1:URosarioX+URX01+1T2020/jum p_to/block-v1:URosarioX+URX01+1T2020+type@sequential+block@674fda5e810440f190d84 9740e674cae' Processing 'https://courses.edx.org/courses/course-v1:URosarioX+URX01+1T2020/jum p_to/block-v1:URosarioX+URX01+1T2020+type@sequential+block@fe847e5e361b47a3a3efd 82f480b2a4e' Processing 'https://courses.edx.org/courses/course-v1:URosarioX+URX01+1T2020/jum p_to/block-v1:URosarioX+URX01+1T2020+type@sequential+block@29c2dfb8e8294eed941ee 3b576db59c8' Removed 0 duplicated urls from 0 in total Output directory: Downloaded