search

rugantio / fbcrawl

A Facebook crawler
Apache License 2.0
661 stars 229 forks source link

IndexError: list index out of range in comments.py line63 #67

Open chrixtal opened 4 years ago

chrixtal commented 4 years ago

when I trying to crawling my site, it occurs error:

  File ".\comments.py", line 63, in parse_page
    temp_post = response.urljoin(post[0])
IndexError: list index out of range

the above line is: post = post.xpath(".//a[contains(@href,'footer')]/@href").extract()

I trying to open source of my target site page, It have 13 'footer', and I check the post[], it len(post[]) is 0.

how come?, is anyone can give me a suggestion?

slakat commented 4 years ago

You're probably using "page" instead of "post", try changing that:

scrapy crawl comments -a email="email@gmail.com" -a password="pass" -a post="https://mbasic.facebook.com/story.php?story_fbid=1212323232" -o trump_comments.csv