search

shaikhsajid1111 / facebook_page_scraper

Scrapes facebook's pages front end with no limitations & provides a feature to turn data into structured JSON or CSV
https://pypi.org/project/facebook-page-scraper/
MIT License
242 stars 67 forks source link

returning only one post #84

Open ihabpalamino opened 1 year ago

ihabpalamino commented 1 year ago

facebook-page-scraper is returning only one whatever the number of postCount i give

shaikhsajid1111 commented 1 year ago

Can you give more details? About the environment, facebook page name and code?

ihabpalamino commented 1 year ago

actually the code is working on a machine but not on 3 others machine windows version: 11 python version:11 or 10 firefox version;116.0(64bit) facebook_page_scraper version : 5.0.2 the page is public that i am trying to scrape and this the code import sys import re from facebook_page_scraper import Facebook_scraper

username="AlAoulaTVMaroc" posts_count=10 browser = "firefox" proxy = "IP:PORT" #if proxy requires authentication then user:password@IP:PORT timeout = 60 headless = True namepage = Facebook_scraper(username, posts_count, browser, proxy=proxy, timeout=timeout, headless=headless)

json_data = namepage.scrap_to_json() encoded_json_data = json_data.encode('utf-8') encoded_json_data = json_data.encode('utf-8', errors='replace')

Use sys.stdout.buffer to write the encoded data to the standard output

sys.stdout.buffer.write( encoded_json_data )

ihabpalamino commented 1 year ago

@shaikhsajid1111 any updates??

shaikhsajid1111 commented 1 year ago

Did you try setting headless to False?. I'm unavailable these days. Might be able to look deeply into this upcoming weekend

ihabpalamino commented 1 year ago

when the headless is false it says that i have to log in while i am already logged in in my browser and it works only when i delete the login form

ihabpalamino commented 1 year ago

here is a screen of what i get when i turn it to false image

shaikhsajid1111 commented 1 year ago

Oh Okay, scraper isn't closing this dialogue automatically?

ZAIRIyoussef commented 1 year ago

Hello, the same issue for me. However I found that the scraper is unable to locate by css selector the element [aria-label="Close"] that's why the sing up dialog doesn't close so the crowler can't scroll down to find more post. The code works for me when I replaced the [aria-label="Close"] by the css selector '.xc9qbxq'

ihabpalamino commented 1 year ago

Oh Okay, scraper isn't closing this dialogue automatically?

no it does not i hope you can fix that or there is already a solution for that