search

rugantio / fbcrawl

A Facebook crawler
Apache License 2.0
661 stars 229 forks source link

Empty page comments file #65

Closed Rahulsunny11 closed 4 years ago

Rahulsunny11 commented 4 years ago

Hi, @rugantio
I ran following command to scrape all comments in Trump page but the output csv file is empty scrapy crawl comments -a email="" -a password="" -a page="DonaldTrump" -a date="2020-04-04" -a lang="it" -o trumpcomments.csv

MaLijia-thss15 commented 4 years ago

I got the same problem.

Rahulsunny11 commented 4 years ago

just change line44 to for post in response.xpath("//article[contains(@data-ft,'top_level_post_id')]"): it will work