search

refeed / scrapy_facebooker

Collection of scrapy spiders which can scrape posts, images, and so on from public Facebook Pages.
MIT License
26 stars 6 forks source link

Add spiders that use Graph API to receive the datas #3

Open refeed opened 7 years ago

refeed commented 7 years ago

Fetch the datas like the list of facebook posts, videos, events is hard with traditional scraping, since we fetch it from the ajax response of a facebook endpoint that we don't know how to use it, we just guessing it (it looks slightly hacky), and the common problem is it's sometimes work and sometimes doesn't.

For the next step we'll deprecate the traditional spiders, and use the official Graph API from Facebook instead.

refeed commented 7 years ago

I think we still need to support the traditional scraper spiders, since it's kinda difficult to get an Graph API key (because we need to be a graph api dev)