Rate limit exceeded

[JustLady]

with api elevated access it pulls nothing and exports no blocks just outputs this

`Twitter Blocklist exporter v0.1 - https://github.com/Forceflow/twitter_blocklist_exporter

Loading Twitter API keys from file Authenticating with Twitter Grabbing blocklist Traceback (most recent call last): File "/home/dawl/twitter_blocklist_exporter/blocklist_exporter.py", line 79, in main() File "/home/dawl/twitter_blocklist_exporter/blocklist_exporter.py", line 73, in main blocklist = get_blocklist_from_API(api) File "/home/dawl/twitter_blocklist_exporter/blocklist_exporter.py", line 51, in get_blocklist_from_API for user in tweepy.Cursor(api.get_blocks, skip_status=True).items(): File "/home/dawl/.local/lib/python3.10/site-packages/tweepy/cursor.py", line 86, in next return self.next() File "/home/dawl/.local/lib/python3.10/site-packages/tweepy/cursor.py", line 286, in next self.current_page = next(self.page_iterator) File "/home/dawl/.local/lib/python3.10/site-packages/tweepy/cursor.py", line 86, in next return self.next() File "/home/dawl/.local/lib/python3.10/site-packages/tweepy/cursor.py", line 110, in next data, cursors = self.method(cursor=self.next_cursor, File "/home/dawl/.local/lib/python3.10/site-packages/tweepy/api.py", line 33, in wrapper return method(*args, *kwargs) File "/home/dawl/.local/lib/python3.10/site-packages/tweepy/api.py", line 46, in wrapper return method(args, **kwargs) File "/home/dawl/.local/lib/python3.10/site-packages/tweepy/api.py", line 3034, in get_blocks return self.request( File "/home/dawl/.local/lib/python3.10/site-packages/tweepy/api.py", line 269, in request raise TooManyRequests(resp) tweepy.errors.TooManyRequests: 429 Too Many Requests 88 - Rate limit exceeded`

not sure tho if the issue is twitter or somethign on my end?

EDIT: now confirmed same issue on windows and linux on separate devices. prolly relevant that my blocklist is up near 20 million too :P

[Forceflow]

Oh yeah, definitely an API limit there. I'll see if I can work in a way to do partial dumps, though your use case is on the extreme side. Sorry!

[JustLady]

ah yea i figured out i can put a time.sleep(1) in there to make it slow enough to be fine. the real issue is that with 15.6 million blocks (i checked :P) that means itd take 150~ days to export the whole list.

the real solution would be using v2 of the api to pull pages of the list per request but idk how to do all that. im just IT, not a programer :/

On Mar 29, 2023 at 6:15 PM, Jeroen Baert @.***> wrote:

Oh yeah, definitely an API limit there. I'll see if I can work in a way to do partial dumps.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[Forceflow]

Yeah, the issue is that V2 is not easily available without paying Twitter through the nose.