search

coursera-dl / edx-dl

A simple tool to download video lectures from edx.org (and other openedx sites)
GNU Lesser General Public License v3.0
1.92k stars 638 forks source link

Download failed with "AttributeError: 'NoneType' object has no attribute 'string'" #593

Open scottgmcleod opened 4 years ago

scottgmcleod commented 4 years ago

Subject of the issue

Download failed with "AttributeError: 'NoneType' object has no attribute 'string'"

Your environment

Steps to reproduce

Tell us how to reproduce this issue. Please provide us the course URL, and the specific subsection or unit if possible. The tool sees the course I am enrolled in at Stanford but when I try to download the url (https://lagunita.stanford.edu/courses/HumanitiesSciences/StatLearning/Winter2016/course/) it fails.

Expected behaviour

It should commence the download

Actual behaviour

Output from edx-dl -x stanford -u username@domain.com https://lagunita.stanford.edu/courses/HumanitiesSciences/StatLearning/Winter2016/course/

edx_dl version 0.1.13 Password: Building initial headers for future requests. Getting initial CSRF token. Found CSRF token. Logging into Open edX site: https://lagunita.stanford.edu/login_ajax Extracting course information from dashboard. Traceback (most recent call last): File "/Users/scottm/miniconda2/bin/edx-dl", line 8, in sys.exit(main()) File "/Users/scottm/miniconda2/lib/python3.6/site-packages/edx_dl/edx_dl.py", line 1028, in main for selected_course in selected_courses} File "/Users/scottm/miniconda2/lib/python3.6/site-packages/edx_dl/edx_dl.py", line 1028, in for selected_course in selected_courses} File "/Users/scottm/miniconda2/lib/python3.6/site-packages/edx_dl/edx_dl.py", line 186, in get_available_sections sections = page_extractor.extract_sections_from_html(page, BASE_URL) File "/Users/scottm/miniconda2/lib/python3.6/site-packages/edx_dl/parsing.py", line 403, in extract_sections_from_html for i, section_soup in enumerate(sections_soup, 1)] File "/Users/scottm/miniconda2/lib/python3.6/site-packages/edx_dl/parsing.py", line 403, in for i, section_soup in enumerate(sections_soup, 1)] File "/Users/scottm/miniconda2/lib/python3.6/site-packages/edx_dl/parsing.py", line 392, in _make_subsections for i, s in enumerate(subsections_soup, 1)] File "/Users/scottm/miniconda2/lib/python3.6/site-packages/edx_dl/parsing.py", line 392, in for i, s in enumerate(subsections_soup, 1)] AttributeError: 'NoneType' object has no attribute 'string'

Ken0sis commented 4 years ago

Exact same issue for me, except I'm on Python 2.7.

scottgmcleod commented 4 years ago

The tool does work for an edX course I took. But not lagunita...

fixfo commented 4 years ago

Same error on lagunita. Ubuntu 19.10, python 3.7.5 (system)

mohan-barathi commented 4 years ago

Same error for me ..! I am using Python 2.7.14 on Windows 10

I can download edx courses ..! But I am facing issue in lagunita.stanford.edu

Is there anyother way to download, as Lagunita is retiring and will shut down at 12 noon Pacific Time on March 31, 2020.

MATRIX30 commented 4 years ago

Same error for me ..! I am using Python 2.7.14 on Windows 10

I can download edx courses ..! But I am facing issue in lagunita.stanford.edu

Is there anyother way to download, as Lagunita is retiring and will shut down at 12 noon Pacific Time on March 31, 2020.

Replace your parsing.py with the one found in the zip file below. parsing.zip

Ken0sis commented 4 years ago

MATRIX30's edit of the parsing.py file seems to work for me. Thank you!

scottgmcleod commented 4 years ago

Yep Worked for me too. Thanks MATRIX30!

ny83427 commented 4 years ago

"ERROR: This video is no longer available due to a copyright claim by Stanford University. Sorry about that."😭

parsing.py works great but failed before copyright claim:(

mohan-barathi commented 4 years ago

@ny83427 comment out line 42 on utils.py, and add a print statement print (e, "move on..!")

Usually introduction videos are with copyrights .. you can download them manually ..!! 🔨 (quick fix..!)

ny83427 commented 4 years ago

@mohan-barathi HOLY COW! This works like a charm. Thank you so much, dude!

ichit commented 4 years ago

Thanks

ny83427 commented 4 years ago

@ny83427 comment out line 42 on utils.py, and add a print statement print (e, "move on..!")

Usually introduction videos are with copyrights .. you can download them manually ..!! 🔨 (quick fix..!)

I do think the workflow of the code can be improved a little bit as you suggested, currently, one failure will stop the whole world, which is the worst scearnio. Continue fetching other resources while output an error list would be more friendly.

RakeshIIITD commented 4 years ago

It's not working for me for lagunita. Same string error is coming even after replacing parsing.py as suggested by MATRIX20. ERROR

edx_dl version 0.1.13 Building initial headers for future requests. Getting initial CSRF token. Found CSRF token. Logging into Open edX site: https://lagunita.stanford.edu/login_ajax Extracting course information from dashboard. Traceback (most recent call last): File "/usr/local/bin/edx-dl", line 10, in sys.exit(main()) File "/usr/local/lib/python2.7/dist-packages/edx_dl/edx_dl.py", line 1028, in main for selected_course in selected_courses} File "/usr/local/lib/python2.7/dist-packages/edx_dl/edx_dl.py", line 1028, in for selected_course in selected_courses} File "/usr/local/lib/python2.7/dist-packages/edx_dl/edx_dl.py", line 186, in get_available_sections sections = page_extractor.extract_sections_from_html(page, BASE_URL) File "/usr/local/lib/python2.7/dist-packages/edx_dl/parsing.py", line 403, in extract_sections_from_html for i, section_soup in enumerate(sections_soup, 1)] File "/usr/local/lib/python2.7/dist-packages/edx_dl/parsing.py", line 392, in _make_subsections for i, s in enumerate(subsections_soup, 1)] AttributeError: 'NoneType' object has no attribute 'string'

ny83427 commented 4 years ago

@RakeshIIITD Replacing parsing.py is not enough.please check @mohan-barathi 's solution:

comment out line 42 on utils.py, and add a print statement print (e, "move on..!")

This works at least for me.

Druadach commented 4 years ago

thanks

numlockkey commented 4 years ago

None pf these solutions work anymore.

mohan-barathi commented 4 years ago

@numlockkey Because the lagunita.stanford.edu is closed on 31-March-2020. That's the reason we were trying the downloads with workarounds, and didn't concentrate on fixing. 😃