search

humandecoded / twayback

Automate downloading archived deleted Tweets.
Apache License 2.0
178 stars 23 forks source link

added async POC for checking url status #4

Closed humandecoded closed 2 years ago

humandecoded commented 2 years ago

Just did a quick proof of concept adding async to the part where you check twitter urls. Seeing a significant speed increase.

Would recommend merging this in to a separate branch for testing purposes.

Mennaruuk commented 2 years ago

Hi! Thanks for your amazing contribution. (YAY!)

I tested it, and it pretty much works great! Speeds are amazing, and I think batches of 15 URLs isn't bad, esp. with it being Twitter.

I did realize that, on Windows, I kept encountering a RuntimeError. I looked it up, and I found a fix here that worked for me.

I will merge! Thanks again, you made my night!!