search

rugantio / fbcrawl

A Facebook crawler
Apache License 2.0
667 stars 229 forks source link

Comment crawler does not work #8

Closed everyl12 closed 5 years ago

everyl12 commented 5 years ago

Hi Rugantio! I am using the comment crawler for comments from FB pages in Chinese (e.g., "govnews.hk"). The comment crawler returned 0 tweet although there should be several comments. But the crawler works well for comments from Trump's page. Could you please help?

everyl12 commented 5 years ago

I found that it was related to the number of comments of a post, but not language. When there were fewer tweets (<20), the comment crawler would return 0 tweets. When there were more comments (e.g., Trump), the comment crawler works well.

rugantio commented 5 years ago

Hi, the main selector in parse_page was not working well, try now. Sorry but the comments spider is just a stub, you're welcome to add/fix functionalities to it :)

everyl12 commented 5 years ago

Thank you very much for the update! I tried but the text was not right. Now nearly every comment contains the same content ("//這亦形成了一個與海耶克所描繪的相反景象.....").

everyl12 commented 5 years ago

Yes I am also trying myself. I am a social science researcher beginning to learn python, crawling, and so on. I find your crawler useful for my research project :) Thank you for your contribution.

gaukhar98 commented 5 years ago

Hi, Rugantio! I am trying to use your comment crawler, but it is not working for me? I wrote on console like this: scrapy crawl comments -a email="EMAIL" -a password="PASSWORD" -a page="DonaldTrump" -o TrumpComment.csv Am I doing write? Or what should we write to the page argument (I am confused)? Could you give me example. Thanks in advance!

rugantio commented 5 years ago

The comments spider was broken, should work now! Let me know if you encounter further problems