Add indexes to improve searches on hashtags

TheDataRideAlongs / ProjectDomino

Scaling COVID public behavior change and anti-misinformation

Apache License 2.0

61 stars 13 forks source link

Add indexes to improve searches on hashtags #66

Closed bechbd closed 4 years ago

bechbd commented 4 years ago

Add indices to allow for searching on hashtags such as:

MATCH (t:Tweet)<-[e:TWEETED]-(a:Account) WHERE SIZE(t.hashtags) > 0 AND t.hashtags =~ "stayathome|quarantinelife" RETURN a.name as name, a.id as account_id, t.hashtags as tags, t.created_at.month as month, t.created_at.day as day, t.id as tweet_id LIMIT 100

lmeyerov commented 4 years ago

Variants:

count(*) as hits ORDER BY hits
focus on 1-4-grams, not just hashtags
when tag isn't provided (top tags overall)
given a community label or set of accounts, get the tags / top n-grams in their tweets

Priority: -- Daily/weekly topic analysis in general -- ... and by large set of accounts -- ... and by inferred community/geo (TBD)

bechbd commented 4 years ago

@lmeyerov Can you clarify the below:

count(*) as hits ORDER BY hits Is this to find something like the most common hashtags by day?
focus on 1-4-grams, not just hashtags This is to find n-grams in the text of the tweet I am assuming here?
when tag isn't provided (top tags overall) Is this limited by day/week or of all time?
given a community label or set of accounts, get the tags / top n-grams in their tweets This sounds like the same requirements requests as above, just limited to a subset of the accounts. Correct?