eleurent
commented
3 years ago Hi @AhmostV,
Yes, these accounts are problematic. The proper way to fix this issue would be to split the getFollowers call into several requests of smaller chunks. I know that SgtSopalin has implemented this, and he should post a pull request soon to add this feature. In
In the meantime, and as a temporary workaround, you can simply exclude these accounts with many followers from the list of fetched users by creating an excluded.json file containing a list of usernames, e.g.:
[
"realDonaldTrump",
"elonmusk"
]They will be skipped in the fetching process, which won"t get blocked.
Afterward, I am under the impression that the credentials used are "black listed" and need to be changed (this is a supposition).
I don't think you get black listed, but the large request just triggers the API rate limit, so you have to wait for 15mn until you can send another request.
FlorentLefebvre
Hello,
Sorry to bother you again, but I am met with a strange problem : Account with large number of followers, the program run forever, and always ends with the "rate limit" error. Afterward, I am under the impression that the credentials used are "black listed" and need to be changed (this is a supposition).
Do you know if there is a hard limit of the number of followers? Or if I used the programm the wrong way?
$ python3 fetch_data.py user eleurent Found 953 followers. Traceback (most recent call last): File "fetch_data.py", line 238, in
main()
File "fetch_data.py", line 222, in main
followers, friends, mutuals, all_users = fetch_users(apis[0], screen_name, search_query,
File "fetch_data.py", line 68, in fetch_users
friends = get_or_set(out_path / friends_file, partial(api.GetFriends, screen_name=user), api_function=True)
File "fetch_data.py", line 155, in get_or_set
value = [item._json for item in value()]
File "/home/thomas/.local/lib/python3.8/site-packages/twitter/api.py", line 2769, in GetFriends
return self._GetFriendsFollowers(url,
File "/home/thomas/.local/lib/python3.8/site-packages/twitter/api.py", line 2664, in _GetFriendsFollowers
next_cursor, previous_cursor, data = self._GetFriendsFollowersPaged(
File "/home/thomas/.local/lib/python3.8/site-packages/twitter/api.py", line 2499, in _GetFriendsFollowersPaged
data = self._ParseAndCheckTwitter(resp.content.decode('utf-8'))
File "/home/thomas/.local/lib/python3.8/site-packages/twitter/api.py", line 4908, in _ParseAndCheckTwitter
self._CheckForTwitterError(data)
File "/home/thomas/.local/lib/python3.8/site-packages/twitter/api.py", line 4928, in _CheckForTwitterError
raise TwitterError(data['errors'])
twitter.error.TwitterError: [{'message': 'Rate limit exceeded', 'code': 88}]