search

minimaxir / download-tweets-ai-text-gen

Python script to download public Tweets from a given Twitter account into a format suitable for AI text generation.
MIT License
222 stars 41 forks source link

Scrape from list #15

Open wradd opened 4 years ago

wradd commented 4 years ago

I've been using your Collab notebook and your blog post about building a text-generating Twitter bot with GPT-2. I would like to train a bot on tweets from an entire Twitter list, as opposed to a single user.

Twint supports scraping a list but doesn't remove replies like this script does, and that's important as you point out in your blog post.

Ideally, a user of this script could use something like this:

python3 download_tweets.py <twitter_list_URL>

This is my first issue or feature request. Very open to feedback!

sdelgadoc commented 4 years ago

The script has been updated to include the ability to scrape tweets from multiple accounts, which are listed in a text file. It isn't as turnkey as your recommendation, but now, you don't have to scrape each Twitter account independently.