IndexError: list index out of range

[sajee]

$ python edx-dl.py Username:
Password:

Traceback (most recent call last): File "edx-dl.py", line 390, in main() File "edx-dl.py", line 222, in main data = soup.find_all('ul')[1] IndexError: list index out of range

[shk3]

It actually works on my computer. Could you provide more information, such as the version of python, soup, and edx-downloader?

[sajee]

I'm using the latest version of edx-downloader.

On Mac 10.8.5

$python -V Python 2.7.2

BeautifulSoup (3.2.1) beautifulsoup4 (4.3.2)

[sajee]

BTW, I forgot to mention that this was working for a few weeks. It broke a couple of days ago.

[abigos]

FYI I've seen this also. I have a script adapted to download from Stanford and that continues to work while the MIT version has started to fail, so I suspect it's a change on the site (which is what the error message points to also). I guess the site could serve different HTML depending on several factors which is why it might not be a simple repro.

[MarwanShehata]

I Have this problem with edx Aerodynamics AE1110x , 1-I installed beautifulsoup

2-youtube-dl (but with an error while trying to update """" Updating to version 2013.11.03 ERROR: no write permissions on youtube-dl """"

3-used cmd as adminstrator

4-turned off the anti-virus program (ESET Smart Security)

and still getting the same problem :(

[vafleite]

Hi, I had the same problem and solved it changing the parser used by BeautifulSoup.

First I installed the lxml library.

At the line 210 I changed from soup = BeautifulSoup(dash) to soup = BeautifulSoup(dash, "lxml")

And now it's working!

[abigos]

That fix works for me also, thanks! I guess a good solution would be to add a new command line option to pass in a parser, in the meantime, this works great. Thanks again for sharing your fix.

[rbrito]

In another project of mine, we've "been there, done that":

https://github.com/coursera-dl/coursera/issues/143

In my very humble opinion, we did the right thing with coursera-dl by not adding a new command line option.

Regards,

Rogério Brito : rbrito@{ime.usp.br,gmail.com} : GPG key 4096R/BCFCAAAA http://cynic.cc/blog/ : github.com/rbrito : profiles.google.com/rbrito DebianQA: http://qa.debian.org/developer.php?login=rbrito%40ime.usp.br

[vafleite]

Hello Rogério, I think that the idea of a zip realease containing all dependencies would be great!

[shk3]

@rbrito @iemejia How about the suggestion from @victorfurtadoleite ?

[rbrito]

Hi.

On Thu, Dec 26, 2013 at 4:34 AM, George Monkey notifications@github.com wrote:

@rbrito @iemejia How about the suggestion from @victorfurtadoleite ?

@shk3, For regular dependencies (read: Python modules), that's a good idea. For web scrapers like youtube-dl, it is not, since Youtube frequently breaks and, essentially, we would have to have a new release of our code every time that youtube-dl releases one fix for youtube.

And it is hard to keep up with youtube-dl and the changes that Youtube are undergoing. For instance, the whole DASH videos recently... I described the situation briefly in my homepage:

http://cynic.cc/blog/posts/2013-10-23-assorted_news/

Other than that, I am favorable to instruct users to use pip.

Rogério Brito : rbrito@{ime.usp.br,gmail.com} : GPG key 4096R/BCFCAAAA http://cynic.cc/blog/ : github.com/rbrito : profiles.google.com/rbrito DebianQA: http://qa.debian.org/developer.php?login=rbrito%40ime.usp.br

[iemejia]

I'm closing this since the discussion moved from the actual issue. If you are still interested in the executable version create a new issue, but I agree with @rbrito in this one creating an executable version is probably more cumbersome than useful because we would require to be all the time checking the latest youtube-dl version and upgrading it. I have mixed feelings about doing such thing pip since youtube-dl doesn't provide an api-like interface (which in my opinion is the nice thing of pip and the whole virtualenv ecosystem), however a requirements file could be nice way to track the project dependencies.