minimaxir / download-tweets-ai-text-gen

Python script to download public Tweets from a given Twitter account into a format suitable for AI text generation.
MIT License
219 stars 41 forks source link

Output csv file just contains the word "tweets," no data #18

Open RalphORama opened 4 years ago

RalphORama commented 4 years ago

I'm confident I set up the project correctly (virtualenv with python 3.6.9, ran pip3 install -r requirements.txt).

When I run the script, I see the following in my terminal for about 10 seconds:

$ python3 download_tweets.py bikinibabelover
Retrieving tweets for @bikinibabelover...
Oldest Tweet:   0%|                                                                             | 0/5776 [00:14<?, ?it/s]

Then the process ends with no error message, and I'm left with the file bikinibabelover_tweets.csv.

$ cat bikinibabelover_tweets.csv
tweets

$ 

Any help with this issue would be appreciated

RalphORama commented 4 years ago

Upon closer inspection (enabling logging, etc.), this may be an issue to do with the way twitter handles accounts with "explicit content." Running a search from:bikinibabelover returns "No Results" even though they have quite a few tweets on their timeline if you visit https://twitter.com/bikinibabelover

minimaxir commented 4 years ago

If the output file is effectively empty, that means it scraped 0 tweets. Dunno how Search handles explicit content.

ajtran303 commented 4 years ago

I am having the same result, a csv file with only "tweets". The account I'm scraping from is not explicit.. It has 109160 tweets, is that too much to scrape from perhaps?

RalphORama commented 4 years ago

Seems like it was an issue with Twitter search itself, the tool is working for me now