New insight: Gender Analysis for Facebook Posts

[ashkerina]

One-liner

What sex are people who have made your post the most popular today?

Full explainer

How does it make the user feel? What is the goal of this insight?

Some posts are popular among women, some other among men. It is really important to know what gender makes the posts popular.

Audience for the insight

Which networks, if any, are excluded from this insight? All except Facebook.

Does this insight serve users with less or more activity? It's for everyone, regardless of activity.

First-run: does this insight show up on a user's first crawl? No

How often this insight runs

• [ ] One-time only, if it's never appeared before
• [ ] First crawl only
• [x] Daily
• [ ] Weekly
• [ ] Monthly
• [ ] Yearly
• [ ] Triggered by a data event

  Headline and Body example

Criteria and logic

Using Graph API we can get posts from one day or week analysing field created_time. After that we get array of all comments of every post and count the number of comments for every post. This way we find the most commented post. Array of comments has field from (type User) that contains person who made this comment. From information about user we get value of field gender. Now we can count the number of men and women and draw the pie chart.

Included elements

• [x] Headline
• [x] Body
• [ ] Header image (image off to the left in side-by-side style insights)
• [ ] Hero image (giant image on top)
• [ ] User(s)
• [x] Post(s)
• [ ] Link(s)
• [ ] Line chart
• [ ] Bar chart
• [ ] Other viz
• [x] Other graphic treatment

[ginatrapani]

I like this insight idea very much, thanks Anna.

Will this insight only count the number of comments a post receives, or will it also consider likes? If ThinkUp counts both likes and comments, the insight will have more data to calculate the totals.

To get started coding on this, first ThinkUp will have to start capturing gender in the tu_users table. The users table doesn't have a field for gender yet, so you must add it. Here's how to modify a table's structure in ThinkUp's database:

https://www.thinkup.com/docs/contribute/developers/howto/modifydb.html

Looking at Facebook, when I edit my profile I see you can set your gender as "Male", "Female", or to a custom word or phrase. To accommodate that, let's make this field a 255 varchar, and it should be able to be null (for networks where ThinkUp doesn't get gender info, like Twitter).

Then, modify the class.User.php file (where the User object is defined) to add the new gender field. Then, modify interface.UserDAO.php and class.UserMySQLDAO.php classes to update and insert the gender field. Make sure you update TestOfUserMySQLDAO functions to assert this field is set correctly.

Finally, in the Facebook crawler, when ThinkUp gets information about users from the Graph API, it should store whatever gender value gets returned into the users table. You can use TestOfFacebookCrawler.php to make sure this works as you expect.

I know I've listed a lot here. Give it a try, and let me know if you run into any roadblocks. Thanks again!

[ginatrapani]

Hi @ashkerina,

Looking at the Facebook Graph API Explorer:

https://developers.facebook.com/tools/explorer/

If you add gender to the list of fields you want to see for a user, you'll get it back. So user ID 800832 is a friend of mine which means I have permission to return his data. If I do a request for:

/800832?fields=id,gender

I get back:

{ "id": "800832", "gender": "male" }

If you look at our FacebookCrawler class, ThinkUp doesn't include gender in the list of fields it requests in fetchUser. Try adding it there:

https://github.com/ginatrapani/ThinkUp/blob/master/webapp/plugins/facebook/model/class.FacebookCrawler.php#L82

And then run TestOfFacebookCrawler. In testFetchUser, assert that gender was returned from the API and stored correctly.

Keep in mind, the tests don't actually query the API, they read data files from disk. TestOfFacebookCrawler::testFetchUser reads this test data file:

https://github.com/ginatrapani/ThinkUp/blob/master/webapp/plugins/facebook/tests/testdata/606837591

You'll have to include gender there for the test to be able to see it.

[ashkerina]

Hi, @ginatrapani

I've already added gender to request in fetchUser when the problem appeared. So I've decided to recreate Facebook app for ThinkUp and everything became well. Now I can get all data that I need for my insight.

[ginatrapani]

Great to hear it! Looks like you're making great progress here https://github.com/ashkerina/ThinkUp/commits/1943-gender-analisis

[ashkerina]

Hi, @ginatrapani

I've alreary finish adding gender field in Facebook crawler and creating tests in TestOfFacebookCrawler for it.

Now I am going to create genderanalysis.php and genderanalysis.tpl into Insightsgenerator. Am I on the right way?

[ginatrapani]

Hi @ashkerina - you're on the right track! Keep going.

One thing we like to do is break down projects into small commits, one for each change. So if you'd like, send me a pull request for the work you've done so far just acquiring the gender field. I will review it and go over it with you. Then, you can work on the insight in a separate commit.

Great work!