search

dscottpi / bible_scraper

Scrapy spider for crawling the bible from bible.com
4 stars 0 forks source link

Index Out Of Range #2

Open mattzab opened 7 years ago

mattzab commented 7 years ago

When trying to run this, an error is produced:

File "/home/matt/Downloads/bible_scraper-master/bible_scraper/spiders/bible.py", line 48, in parse
    book = response.xpath('//a[@id="reader_book"]/text()').extract()[0]
IndexError: list index out of range
dscottpi commented 7 years ago

Hey @mattzab. What command did you run? I'll try reproduce and fix.

Thanks!

mattzab commented 7 years ago

Hey @dscottpi, scrapy crawl bible -a version=1171

I changed a few lines in Bible.py, and got it to error out a little later in the program, but still didn't have success. I don't have the changes currently, but I suspect Bible.com changed, which is what broke things. (I think. This is my first time trying to crawl anything with Scrapy)

dscottpi commented 7 years ago

@mattzab yup you're right, this is probably what's wrong. I'll take a look this weekend and update the spider :)

mattzab commented 7 years ago

@dscottpi,

Did you get it fixed? Should I close this issue as solved or leave it open?

dscottpi commented 7 years ago

@mattzab, I'm working on a new version atm, I'll update & close this issue when it's done :)

mattzab commented 6 years ago

@dscottpi,

Any updates as of late?