mchozhang / SinsOnTwitter

4 stars 3 forks source link

Add indexing and processing function to the collector #41

Open Arufu opened 5 years ago

Arufu commented 5 years ago

will built an extra database: index_database, to store ids of tweets that contain keywords. It will be formatted as: {"keyword1":[id1, id2, id3...], "keyword2":[id1, id4, id5], ...} will also write extra fields into existing tweets, e.g. sentiment update: now will do the processing during streaming (not searching, as searching is too fast)