kevinzg / facebook-scraper

Scrape Facebook public pages without an API key
MIT License
2.43k stars 631 forks source link

Advice on how to scrape 7 years of posts from a single public profile without getting banned? #1011

Open Strider3000 opened 1 year ago

Strider3000 commented 1 year ago

I'm attempting to get seven years worth of posts (around 4000 posts) from a single public profile without getting banned. I don't need comments or reactions - just the basic information. I've never made it longer than about 100 days worth at this point without getting temporarily banned, and I'm wondering what techniques I should use to get facebook-scraper to get me all seven years worth of posts?

Here is the command that I'm using:

facebook-scraper -c cookies.json -p 500 -fmt json -s 60 -f profile_output.json -d 2100

Any ideas or input is appreciated. I'm fine using Python as well if that gives the flexibility that I need to accomplish my goal.

cedriferitan commented 1 year ago

from what I've noticed from other issues, this isn't possible with the CLI. you need to write the python script yourself and add a pause interval between requests

Shohreh commented 1 year ago

What about connecting through eg. Tor, without logging on?

FB would ban someone elses's IP :-p

vitovt commented 1 year ago

FB ban account related to your cookies, not IP. TOR and other public proxy IPs are suspicious by default and banned very quickly Without cookies you are very limited on scrapping. You receive less posts, in different order and harder limits.

Shohreh commented 1 year ago

Good to know

yasavvym8 commented 1 year ago

does facebook temporarily ban your actual account? or just your ip?

Shohreh commented 1 year ago

I don't know. I put that on the backburner until I find a safer way to grab posts.

Callum681 commented 1 year ago

Hi, did anyone make any progress with this issue? I'm having the same problem now