search

moda20 / facebook-scraper

Scrape Facebook public pages without an API key
MIT License
88 stars 28 forks source link

Issues getting group listings: getting some listings, not all. URL parameters don't seem to work. #56

Open aoiotoko opened 6 months ago

aoiotoko commented 6 months ago

Hi. I'm trying to fetch the latest item listings from a FB group in chronological order but for some reason I seem to get random listings. They do appear in chronological order, but they're definitely not all of them. It's like I'm getting one or two from each of the past few days, but not even the ones from the current day, it seems really random.

I've tried to fix the issue by adding ?sorting_setting=CHRONOLOGICAL_LISTINGS and ?v=timeline to the start_url inside get_posts() to no avail (those parameters seem to make no difference).

Does anyone have any suggestions?

Thanks!

aoiotoko commented 6 months ago

Here's the code that does the fetching:

for post in get_posts(group="utahboardgamexchange", base_url="https://mbasic.facebook.com", start_url="https://mbasic.facebook.com/groups/utahboardgamexchange/?sorting_setting=CHRONOLOGICAL_LISTINGS", pages=5, options={
        "posts_per_page": 200
    }):
    print(post['text'])
gavinnn101 commented 4 months ago

Has anyone had any luck figuring this out? Running into the same issue with ?sorting_setting=CHRONOLOGICAL working on the desktop facebook site but the mobile/mbasic site doesn't respect it..