minimaxir / facebook-page-post-scraper

Data scraper for Facebook Pages, and also code accompanying the blog post How to Scrape Data From Facebook Page Posts for Statistical Analysis
2.12k stars 663 forks source link

HTTP Error 403: Forbidden #104

Open yasersakkaf opened 6 years ago

yasersakkaf commented 6 years ago

Why am I getting this?

Scraping cnn Facebook Page: 2018-03-22 13:05:48.534324

HTTP Error 403: Forbidden
Error for URL https://graph.facebook.com/v2.4/cnn/posts/?limit=100&access_token=1679745815446543|x-Y0ET5OXgMmn4zv75e_KZgcEFw&since=2015-01-01&until=2018-03-01&fields=message,link,created_time,type,name,id,comments.limit(0).summary(true),shares,reactions.limit(0).summary(true): 2018-03-22 13:05:55.654418
Retrying.
HTTP Error 403: Forbidden
Error for URL https://graph.facebook.com/v2.4/cnn/posts/?limit=100&access_token=1679745815446543|x-Y0ET5OXgMmn4zv75e_KZgcEFw&since=2015-01-01&until=2018-03-01&fields=message,link,created_time,type,name,id,comments.limit(0).summary(true),shares,reactions.limit(0).summary(true): 2018-03-22 13:06:01.665427
bethke commented 6 years ago

I'm getting the same and am wondering if it is because "Access to certain types of API data is paused for non-active accounts due to changes we are making to the Facebook Platform" It looks like that just started 20 hours ago...

https://developers.facebook.com/status/issues/205942813488872/

SaschaMai commented 6 years ago

Hey, thanks for your great script! Unfortunately I am getting the same error. Does anyone know a fix, yet?

"HTTP Error 403: Forbidden Error for URL https://graph.facebook.com/v2.9/cnn/posts/?limit=100&access_token=censored&fields=message,link,created_time,type,name,id,comments.limit(0).summary(true),shares,reactions.limit(0).summary(true): 2018-03-23 21:36:03.373614 Retrying."

EDIT: Just read @bethke s link. But what do they mean with non-active accounts? When is my account classified as non-active?

marwann commented 6 years ago

This is something apparently related to the Cambridge Analytica scandal. Facebook temporarily blocks API access to some data to avoid such leaks. I think all we have to do is be patient...

SaschaMai commented 6 years ago

@marwann Thanks, man.

rlorenz123 commented 6 years ago

Does anyone have an update on this or know how long Facebook may block API access? I am asking because I need Facebook user post data in order to write my MA thesis.

marwann commented 6 years ago

Works again for me @rlorenz123 - Do you have an old app token that you could try and use?

luckyankit commented 6 years ago

same error comes when you use "user access token" of profile, not of any app. Seems like all old apps are blocked for accessing any data!

rlorenz123 commented 6 years ago

No, unfortunately, I do not have an old app token to use because I just got started a couple of weeks ago.

luckyankit commented 6 years ago

Yes, new ones are also blocked currently. Only a few which are manually approved by FB are working. I wish if I had any approved one!

EkanshdeepGupta commented 6 years ago

Hi. I am also getting the same error, ie "HTTP Error 403: Forbidden".

I only started today, so I don't have an old or any approved token. Any suggestions on what I can do?

mw0000 commented 6 years ago

the code still works, but you need to make some changes

  1. log in to fb
  2. go to https://developers.facebook.com/tools/explorer/
  3. copy the access token value
  4. edit a *.py file you want use in your project (e.g. get_fb_posts_fb_page.py):
    • comment a line with access_token = app_id + "|" + app_secret
    • define new token value:

access_token = "<YOUR TOKEN VALUE>"

  1. save & run - it should work now
  2. the token can expire - so in a few days when you want to use the script again please regenerate token and paste a new value into the code
EkanshdeepGupta commented 6 years ago

Thank you so much, that worked flawlessly!

moonlightmelody commented 6 years ago

Could it be that it isn't possible to scrape links anymore? I tried this updated version and I'm only getting the postings that are videos, pictures and regular status updates, but no more links for posts from early 2017.

jayassingh commented 6 years ago

@mw0000 this works but the key expires in an hour and I am unable to get all the comments from a page.

karimbk commented 6 years ago

Hey guys! Is someone scraping facebook pages feed posts? Why am I getting not all posts returned? Am I doing it wrong, or it is not possible to get all the feed posts of a specific page? Please help, I am writing my Bachelor Thesis on that, and getting just 5-10% of the pages' posts is really getting me nervous!