taspinar / twitterscraper

Scrape Twitter for Tweets
MIT License
2.39k stars 581 forks source link

0 result returned in some certain cases #315

Closed zhicheng0501 closed 4 years ago

zhicheng0501 commented 4 years ago

The search result of Wuhancoronavirus and nCoV returns 0 tweets while the search result of Trump looks normal(returns some certain number of tweets). Does anyone know why it occurs this way? Is it caused by twitter server blocking Wuhancoronavirus and nCoV?

Here are the results of three keywords as follows:

  1. bogon:ncov zhaoningning$ twitterscraper ncov --lang de --limit 100000000 -bd 2020-05-31 -ed 2020-06-01 -o wuhan05310601.json INFO: {'User-Agent': 'Mozilla/5.0 (Windows NT 5.2; RW; rv:7.0a1) Gecko/20091211 SeaMonkey/9.23a1pre', 'X-Requested-With': 'XMLHttpRequest'} INFO: queries: ['ncov since:2020-05-31 until:2020-06-01'] INFO: Querying ncov since:2020-05-31 until:2020-06-01 INFO: Scraping tweets from https://twitter.com/search?f=tweets&vertical=default&q=ncov%20since%3A2020-05-31%20until%3A2020-06-01&l=de INFO: Using proxy 94.153.224.194:58713 INFO: Retrying... (Attempts left: 50) INFO: Scraping tweets from https://twitter.com/search?f=tweets&vertical=default&q=ncov%20since%3A2020-05-31%20until%3A2020-06-01&l=de INFO: Using proxy 154.127.120.18:30280 INFO: Retrying... (Attempts left: 49) INFO: Scraping tweets from https://twitter.com/search?f=tweets&vertical=default&q=ncov%20since%3A2020-05-31%20until%3A2020-06-01&l=de INFO: Using proxy 1.20.97.238:31769 INFO: Retrying... (Attempts left: 48) INFO: Scraping tweets from https://twitter.com/search?f=tweets&vertical=default&q=ncov%20since%3A2020-05-31%20until%3A2020-06-01&l=de INFO: Using proxy 13.235.136.89:80

  2. bogon:wuhancoronavirus zhaoningning$ twitterscraper Wuhancoronavirus --lang de --limit 10000000000000000 -bd 2020-04-27 -ed 2020-04-28 -o wuhan04270428.json INFO: {'User-Agent': 'Mozilla/5.0 (Windows; U; Windows NT 6.1; rv:2.2) Gecko/20110201', 'X-Requested-With': 'XMLHttpRequest'} INFO: queries: ['Wuhancoronavirus since:2020-04-27 until:2020-04-28'] INFO: Querying Wuhancoronavirus since:2020-04-27 until:2020-04-28 INFO: Scraping tweets from https://twitter.com/search?f=tweets&vertical=default&q=Wuhancoronavirus%20since%3A2020-04-27%20until%3A2020-04-28&l=de INFO: Using proxy 94.153.224.194:58713 INFO: Retrying... (Attempts left: 50) INFO: Scraping tweets from https://twitter.com/search?f=tweets&vertical=default&q=Wuhancoronavirus%20since%3A2020-04-27%20until%3A2020-04-28&l=de INFO: Using proxy 154.127.120.18:30280 INFO: Retrying... (Attempts left: 49) INFO: Scraping tweets from https://twitter.com/search?f=tweets&vertical=default&q=Wuhancoronavirus%20since%3A2020-04-27%20until%3A2020-04-28&l=de INFO: Using proxy 1.20.97.238:31769 INFO: Retrying... (Attempts left: 48) INFO: Scraping tweets from https://twitter.com/search?f=tweets&vertical=default&q=Wuhancoronavirus%20since%3A2020-04-27%20until%3A2020-04-28&l=de INFO: Using proxy 13.235.136.89:80

  3. bogon:~ zhaoningning$ twitterscraper trump --lang de --limit 10000000000000000 -bd 2020-05-31 -ed 2020-06-01 -o ncov05310601.json INFO: {'User-Agent': 'Mozilla/5.0 (Windows; U; Windows NT 6.1; x64; fr; rv:1.9.2.13) Gecko/20101203 Firebird/3.6.13', 'X-Requested-With': 'XMLHttpRequest'} INFO: queries: ['trump since:2020-05-31 until:2020-06-01'] INFO: Querying trump since:2020-05-31 until:2020-06-01 INFO: Scraping tweets from https://twitter.com/search?f=tweets&vertical=default&q=trump%20since%3A2020-05-31%20until%3A2020-06-01&l=de INFO: Using proxy 185.157.161.11:8118 INFO: Scraping tweets from https://twitter.com/i/search/timeline?f=tweets&vertical=default&include_available_features=1&include_entities=1&reset_error_state=false&src=typd&max_position=TWEET-1267243681393848322-1267244458866692096&q=trump%20since%3A2020-05-31%20until%3A2020-06-01&l=de INFO: Using proxy 189.127.106.16:53897 INFO: Scraping tweets from https://twitter.com/i/search/timeline?f=tweets&vertical=default&include_available_features=1&include_entities=1&reset_error_state=false&src=typd&max_position=thGAVUV0VFVBaEwLWVwNqTliMWgMCm5eCHlJYjEjUAFQAlAFUAFQAA&q=trump%20since%3A2020-05-31%20until%3A2020-06-01&l=de INFO: Using proxy 188.40.183.184:1080