Closed mpeter50 closed 1 year ago
Could you provide the dump file?
Sure! Here it is: https://gist.github.com/mpeter50/fa6109e32cc87b80df2a07d4ddc0013c The dump file (along with the above exception) was created while trying to scrape the list of tweets from a popular twitter account.
What is the output of snscrape --version
? This looks like a guest token disk thing to me, but the lines don't match up with the development version. Are you on the release version?
If it is a guest token disk failure, you're on an older version, as error handling was added 6 months ago (https://github.com/JustAnotherArchivist/snscrape/blame/46a603053cfbc0ce3c54d43d7e1ac2427fa82b4d/snscrape/modules/twitter.py#L569)
The command says I have this version: snscrape 0.4.3.20220106
According to the tags, this seems to be the latest one.
I installed it with regular pip3 install snscrape
.
Though what you just said made me want to ask this: is it supported to run multiple instances of snscrape at the same time?
Yeah, this fix is currently only in the development version. I intend to make a new release soon.
As a quick fix, you can delete the ~/.cache/snscrape/cli-twitter-guest-token.json
file.
Closing as a duplicate of #494.
Yes, running multiple instances is fine, although the current release version also has a crash bug you might see rarely (#414). In that case, just rerun the scrapes that crash like that.
For the last few days I get the following error every time I try to scrape a tweet (with twitter-tweet scraper) or a list of tweets for a user (with twitter-user scraper):
Just a few days ago (~4) this was working fine, but since then it doesn't. I've searched the issues, and found those about the instagram and facebook scraper, where it was suggested that this is possibly because the user has been ratelimited. But at the same time, I'm able to view the profiles and tweets I try to scrape through twitter.com, even without logging in, so I'm not sure if thats the case now.
Before submitting this issue, I checked if it works on an other machine (of the same network) where it also worked previously, but it still works there.