aronwc
commented
9 years ago Let tweets
We want to create a labeled dataset to predict whether the next tweet a user posts will contain one of the anti-government keywords. We can do this as follows:
For each user U:
- Identify the tweet T that is U's first use of one of the hashtags
- Collect all tweets U posted prior to T. Call this T_u. This becomes a positive example.
- To construct a negative example, remove from T_u the 10 tweets that occurred just prior to T.
To construct a feature vector from a list of a user's tweets, concatenate all tweets together and use TfidfVectorizer, as we did in the ecig experiments.
Here's an example:
User 1:
Tweet 1: a
Tweet 2: a
Tweet 3: b
Tweet 4: b
Tweet 5: c
Tweet 6: c
Tweet 7: d
Tweet 8: d
Tweet 9: e
Tweet 10: e
Tweet 11: f
Tweet 12: #dilmaout
Tweet 13: gThe positive example will consist of tweets 1-11, while the negative example will only contain tweet 1.
We may have to play with how we select the negative tweet, depending on how many tweets each user has (e.g., if many users have fewer than 10 tweets, then we obviously can't construct a negative example). Instead, we can consider using a percentage (e.g., the negative example as 10% fewer tweets than the positive example).
ElaineResende
Predict whether a user will tweet one of the anti-dilma keywords for the first time, based on their prior tweets.