search

Data4Democracy / far-right-analysis

Analysis related to the behavior of extreme far right online communities
35 stars 10 forks source link

Build a dataset of scraped gab.ai data #17

Closed gati closed 7 years ago

gati commented 7 years ago

@hadoopjax and @alejandrox1 have been working on a gab.ai scraper (contact them for details). This issue to take that scraper for a spin, pull down some data, put it into a csv file or sqlite database, and (ideally) publish it on data.world.

If there's time, you could do some exploratory analysis of the dataset. Word clouds, topic modeling, social media analytics like post counts per user, per day. Popular hours of the day to post, etc.

bmcfee commented 7 years ago

I could take a stab at this today.

bmcfee commented 7 years ago

update: scraper works, but is slow. output goes to sqlite with four tables: posts, users, tags, and urls.

I'll leave it to @hadoopjax @alejandrox1 to merge/release when there's enough data pulled to be useful.

gati commented 7 years ago

Thanks for working on this @bmcfee!

alextesy commented 6 years ago

Hi, there is no private messaging on github anymore, so this the best way to contact @hadoopjax and @alejandrox1. I am doing some academy research on gab.ai and I would be really happy to use your scraper or the data that you scraped (if there is any). Thanks a lot!

bstarling commented 6 years ago

I haven't seen either of them around lately. Not sure the gab scraper/results was ever posted here.

alextesy commented 6 years ago

Thanks for the answer! @bstarling

MastaBaba commented 6 years ago

Is the code available for reuse?