search

minimaxir / download-tweets-ai-text-gen

Python script to download public Tweets from a given Twitter account into a format suitable for AI text generation.
MIT License
221 stars 41 forks source link

Google Colab GPT-2 tweets no different than the input #31

Closed walter-grace closed 4 years ago

walter-grace commented 4 years ago

I seem to load the csv file from using the python code, I am using 1000 tweets and the only results are just printing out tweets I gave it vs running putting out GANs tweets. Do I maybe need to use more tweets?

sdelgadoc commented 4 years ago

Yup, 1,000 tweets is on the low side. There is no hard and fast rule, but I’ve heard folks get decent results once they reach 1MB size, which is about 30,000 tweets.

walter-grace commented 4 years ago

Yup, 1,000 tweets is on the low side. There is no hard and fast rule, but I’ve heard folks get decent results once they reach 1MB size, which is about 30,000 tweets. @sdelgadoc I Will try larger dataset, do you know if this py file work by itself? Id love to run this in colab with a GPU for faster trainning

sdelgadoc commented 4 years ago

Yes, this script could run in a Colab. However, I doubt you’ll see an increase in performance. The performance constraint is Twitter’s data transfer rate not processing power. So, I don’t believe that adding a GPU will make it run faster.

walter-grace commented 4 years ago

Yes, this script could run in a Colab. However, I doubt you’ll see an increase in performance. The performance constraint is Twitter’s data transfer rate not processing power. So, I don’t believe that adding a GPU will make it run faster.

Wonderful thank you for the help