search

Jefferson-Henrique / GetOldTweets-python

A project written in Python to get old tweets, it bypass some limitations of Twitter Official API.
MIT License
1.35k stars 809 forks source link

Too few tweets returned #132

Open R2Metoo opened 6 years ago

R2Metoo commented 6 years ago

Hi everyone

I have a slight issue with the script. It seems to return too few tweets when I run it. For example, if I run the following code from the terminal:

~/Desktop/GetOldTweets-python-master$ python Exporter.py --querysearch "refugees" --maxtweets 5000

I would expect to get the maximum amount, since the topic should generate at least 5000 tweets. However, the output file only has 2605 different tweets. Does anyone have an explanation for this problem?

Thanks in advance.

saibunnsi commented 6 years ago

Well, maybe you should narrow down your choices or set a higher maxtweets number and see if you can get more?For example,

tweetCriteria = TweetCriteria().setQuerySearch('ふわふわ').setSince("2013-07-26").setUntil(
 "2013-07-27").setLang('ja').setMaxTweets(1000000)

and it returns more than 6000 tweets? I just doubt that the "maxtweets" is not what we think about it. And it's obvious that we are limited to get more even though we can dig the old ones out. Because, at least in my case, the results was covered by the same tweet after getting a whole 100000 tweets in one day. It turns out that we still need proxies to avoid the limitations... Anyway, @Jefferson-Henrique, thanks!!!

tredmill commented 6 years ago

@saibunnsi can you elaborate on what you mean by proxies? I am facing some sort of extraction limit based on query search and am looking for solutions. In my case the extraction yielded over 24k tweets, but no more (with the same query).