twintproject / twint

An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.

MIT License

15.69k stars 2.71k forks source link

Twint crashing? #619

Closed cad-x2 closed 4 years ago

cad-x2 commented 4 years ago

I'm running the following command to get all tweets from Frankfurt;

twint -g="50.212945,8.715222,10km" --hashtags --csv -o f1.csv --stats -ho --location &

I periodically get the following in the console: CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)

I'm running on ubuntu 18.04 and python 3.6

The issue is that the processes dies after a while. Is there anything that I could do to catch all live tweets?

pielco11 commented 4 years ago

Issue already reported

A workaround is available in the latest version of Twint, there's not a stable solution as of now

pielco11 commented 4 years ago

604

cad-x2 commented 4 years ago

I don't think that was the issue, the same problem persists. Oddly, I'm scrapping tweets from various cities around the world, and always twint stops very quickly (after less than 3-4 thousand tweets) in Frankfurt twint -g="50.212945,8.715222,10km" --hashtags --csv -o f1.csv --stats -ho --location &

pielco11 commented 4 years ago

May you post here or on PasteBin the content of twint-last-request.log, please?