search

lintool / warcbase

Warcbase is an open-source platform for managing analyzing web archives
http://warcbase.org/
161 stars 47 forks source link

Automatic topic issues classifier #151

Open lintool opened 8 years ago

lintool commented 8 years ago

@ianmilligan1 Do you know about this? http://www.policyagendas.org/page/topic-codebook

Do you think it'd be useful to build a topic classifier and integrate it in Warcbase?

The classifier would take in a blob or text and spit out a label (issue area in the codebook above), which you could then filter and group by. So, for example, you might compare (via a word cloud?), the platforms of different parties on defense, over time.

The codebook is a bit US-centric though, but dunno if it matters...

ianmilligan1 commented 8 years ago

This looks great - I think this'd be useful, and these topics look like they'd complement some of our collections really well: the CPP collection we've got at http://webarchives.ca, and then the Canadian/American collections we'll have available for the hackathon. I don't think the US-centric nature is that big a deal, tho we can take a closer look.