search

hhsm95 / FacebookPostsScraper

Scraper for posts in Facebook user profiles, pages and groups
MIT License
136 stars 54 forks source link

Increase number of scraped posts #4

Open danielk56 opened 4 years ago

danielk56 commented 4 years ago

Hi,

Right now when I scrape, only 7 posts are returned. I was wondering if there is a way to increase the number of posts scraped

m07med176 commented 4 years ago

there is no any way to collect more posts or even i have ability to control on number of posts i get bcs this scripts is meaningless without this param

robman1992 commented 4 years ago

Yes, please let us know how to increase number of scraped posts ;)

Aidin12 commented 4 years ago

hi Hugo, Is there a way to add an Infinite Scroll function? I have done an infinite scroll using nodejs but I am not sure how to do it with Python. I have the same issue - I can only scrape 5 posts. I would like to scrape more. So far, it works well - thank you for a great tutorial! Muchas gracias.

-A

Aidin12 commented 4 years ago

we may need to use browser.get; with nodejs we use document.body.scrollHeight. How is this function written with Python?

hhsm95 commented 4 years ago

Okey, let's talk about this functionality. This code only gets the last 5 posts of the profile because they are the ones that appear on the timeline, this in many cases is enough if you only want to monitor a profile. Previously, I have extracted older posts, but because I use a User Agent from an older mobile phone, Facebook limits the amount of historical posts that can be viewed in this kind of User Agent, so you cannot extract all posts from a profile. My code doesn't use Javascript to render profiles, it is very basic only to monitor profiles. In the next few days, when I have some time, I will apply a change to extract all the posts that the mobile version of Facebook allows my User Agent. Thank you all for the comments.

rytmisk commented 4 years ago

Hi I do a research project where I need to extract several thousand posts from a group. I have tried the Ultimate Facebook Scraper which works, but only for around 1700 posts due to javascripts slowing down the search (probably). Your output is really neat so I would be very happy if it was possible to use this script.

gvrkiran commented 3 years ago

is this still in the plan? to be implemented? this will be really useful for me too!