Implement Twitter Service

[Minitour]

View the following wiki page and implement the service https://github.com/Minitour/BigDataHW4/wiki/Twitter-Service

[aPaXuC]

Uploaded get_all_tweets notebook

Changed query_data to: [('language', 'en'),('follow','2669983818,63299591,6529402')]

Inside follow you should provide ID of accounts you want to read.

For now you can use this service to convert usernames to IDs: https://tweeterid.com/

Right now collecting tweets from 3 accounts with high activity:

2669983818 - https://twitter.com/test5f1798 63299591 - https://twitter.com/VENETHIS 6529402 - https://twitter.com/notiven

[Minitour]

@aPaXuC that's actually a pretty good idea, to get the "friends" of a user who is following active users. Since there is no official API to "most followed users" that could be a good work around. What I had in mind was to fetch the top 500 from social blade (see issue #1).

[Minitour]

@aPaXuC also here is the api which converts usernames to user ids:

https://developer.twitter.com/en/docs/accounts-and-users/follow-search-get-users/api-reference/get-users-lookup

[aPaXuC]

I think, that conversion should be a part of making csv/json file, which will contain 2 columns: username | user_id And then I will get only second column and work with it.

Also I'm not sure if we should follow after 500 most important accounts:

• We are using GET request in order to subscribe to tweets of selected users
• Maximum length of URL is ~ 2048 chars, but I'm not sure if we have some other limitations over twitter API

So maybe we need to 'follow' less accounts to be sure that we get all their tweets

[Minitour]

The query parameter length limit is a browser only limit. Perhaps twitter severs allow more than that.

See https://stackoverflow.com/questions/812925/what-is-the-maximum-possible-length-of-a-query-string