minimaxir / facebook-page-post-scraper

Data scraper for Facebook Pages, and also code accompanying the blog post How to Scrape Data From Facebook Page Posts for Statistical Analysis
2.12k stars 663 forks source link

Error, in "get_fb_comments_from_fb" #119

Open alexis2281 opened 6 years ago

alexis2281 commented 6 years ago

I have this error, with "get_fb_comments_from_fb" screenshot_5

Traceback (most recent call last): File "C:\Users\fn.CP-0394753\Desktop\Scrap2\get_fb_comments_from_fb.py", line 231, in if name == 'main': scrapeFacebookPageFeedComments(file_id, access_token) File "C:\Users\fn.CP-0394753\Desktop\Scrap2\get_fb_comments_from_fb.py", line 160, in scrapeFacebookPageFeedComments comment, status['status_id']) File "C:\Users\fn.CP-0394753\Desktop\Scrap2\get_fb_comments_from_fb.py", line 93, in processFacebookComment comment_author = unicode_decode(comment['from']['name']) KeyError: 'from'

lvsanalytics commented 6 years ago

The error means that it is failing to pull the ['from']['name'] field out of the dictionary. This is failing because that information isn't being returned when you make the request. Why?

Most likely because you are using an application or user access token and not a page access token. Those fields are only returned when you use the page access token.

I switched over to a page access token for a page I am an administrator on and it is working fine for me. Now I need to get a permanent access token to run against. You can find more on access tokens here.

https://developers.facebook.com/docs/facebook-login/access-tokens#pagetokens

Easiest solution is to get a permanent token for your page and create a version of the get_fb_comments_from_fb.py for each page you want to pull comments from (probably do this for get_fb_posts_fb_page.py. too and use the page access token.

If you want comments for a page you don't own, a temporary solution that worked for me was to go to line 91 and switch comment['']['name'] to a temporary value, something like ('unknown')

comment_author = unicode_decode(comment['from']['name']) comment_author = unicode_decode('test')#(comment['from']['name'])

You won't get the author information but you will get everything else. I need to look more into it and see if there is another way to get the comment author info.

alexis2281 commented 6 years ago

Thanks. It works

quyleanh commented 6 years ago

@jcommaroto the link is seem broken? Could you give the correct link? Thank you!

GET /me/accounts HTTP/1.1 Host: graph.facebook.com

lvsanalytics commented 6 years ago

I believe that is because they are up to 3.1 now. Goto the Graph explorer

https://developers.facebook.com/tools/explorer/

You can get a Page Access token here