kevinzg / facebook-scraper

Scrape Facebook public pages without an API key
MIT License
2.4k stars 627 forks source link

To fetch the information of users engaged in a post #158

Open rowedenny opened 3 years ago

rowedenny commented 3 years ago

Thanks for the work.

Following collecting the number of likes/comments/shares, I would like to collect the demographic information of the users who engaged in a post. More specifically, given a post with 98 likes, I would like to investigate the age and gender distribution of them. Thus I assume I need to firstly identify the users engaged in a post, and tracing back to their information, and check if there are publicly available gender and year of birth.

So I am wondering if it is possible to make it, given I can only get access to the profile information publicly available.

kevinzg commented 3 years ago

I don't think Facebook freely publish that information.

neon-ninja commented 3 years ago

This PR (https://github.com/kevinzg/facebook-scraper/pull/201) in conjunction with this PR (https://github.com/kevinzg/facebook-scraper/pull/188, already merged) could make this possible (for commenters, not likers). Sample code:

posts = list(get_posts(post_urls=["https://m.facebook.com/story.php?story_fbid=1917763655047425&id=285708024919671"], cookies="cookies.txt", options={"comments":True}))
for commenter in posts[0]["comments_full"]:
    account = commenter.get("commenter_url").replace("https://facebook.com/", "").split("?")[0]
    profile = get_profile(account, cookies="cookies.txt", timeout=15)
    print(account, profile.get("Basic info"))

returns:

jess.anshaw {'Gender': 'Female'}
christine.salt.14 {'Gender': 'Female'}
paris.atkins15 None
Nephtis None
hannah.styles.96 {'Gender': 'Female'}
rebekah.nicholson.3 {'Gender': 'Female'}
jessica.holt.393 {'Gender': 'Female'}
alexwhitedgm {'Gender': 'Male'}
chelsie.foster.5 {'Gender': 'Female', 'Languages': 'English language'}
steven.gigacz {'Gender': 'Male'}
neon-ninja commented 3 years ago

As of https://github.com/kevinzg/facebook-scraper/pull/194#issuecomment-815325534 you can now extract users reacting to a post. Note that Like is just one of 7 possible reaction types.

ambrel5 commented 3 years ago

Hi there, I keep getting an error when running the above code:

image

What am I doing wrong?

Edit: To me, it seems that I am not handling cookies right, therefore the array comments_full is empty.

neon-ninja commented 3 years ago

yeah, if you're not logged in / passing cookies, commenter_url will always be None