Open MadAim123 opened 1 year ago
Should be fixed, now if there are no working instances it will end the scraping without errors.
Now there is another infinity loop:
Current stats for Bildung+vernetzt: 3 tweets, 0 threads...
Current stats for Bildung+vernetzt: 3 tweets, 0 threads...
Current stats for Bildung+vernetzt: 3 tweets, 0 threads...
Current stats for Bildung+vernetzt: 3 tweets, 0 threads...
Until it reaches the max number of retries or the IP is blocked: All instances are unreachable. Check your request and try again.
Can you try it for your own for the term "Bildung vernetzt"? Parameters:
The created link would be: https://nitter.ktachibana.party/search?f=tweets&q=Bildung+vernetzt&since=2022-12-25
The first tweet is always visible (on every page after click on "Load more").
Perhaps they would stop going down if you stopped scraping them.
This scraper does not abuse the instances. I limited it to 1 request at least every 2 seconds, you can see it in the code that it sleeps for 1-2 seconds on every search and then another 2 seconds before every retry. I hope such a low request frequency is not an issue.
Same problem with infinite testing on version 0.2.9
scraper.get_tweets('#TESLA', mode='hashtag', number=256)
I always get this message after a while:
High number of retries detected. Testing all instances... New working instances:
It's an infinity loop. Only breaking the script works. There is no new working instances named (and used). Only thousand times the same message.