search

taspinar / twitterscraper

Scrape Twitter for Tweets
MIT License
2.41k stars 579 forks source link

It doesn't pull out tweets #148

Open Camilo-Lesmes opened 6 years ago

Camilo-Lesmes commented 6 years ago

Good afternoon, today I searched several times and I couldn't find anything. I'm using the 3.5.2 Python's version with Django 1.11 and I haven't had any error. This is how I searched:

twitterscraper bancolombia -bd 2018-10-01 -ed 2018-10-22 --poolsize 30 -o 93787760.json

INFO: Got 0 tweets (0 new). INFO: Got 0 tweets for bancolombia%20since%3A2018-10-05%20until%3A2018-10-06. INFO: Got 0 tweets (0 new). INFO: Got 0 tweets for bancolombia%20since%3A2018-10-12%20until%3A2018-10-13.

twitterscraper "Gipsy Kids Festival" -bd 2018-06-01 -ed 2018-09-27 --poolsize 30 -o 92342530.json

INFO: Got 0 tweets for Gipsy%20Kids%20Festival%20since%3A2018-07-18%20until%3A2018-07-22. INFO: Got 0 tweets (0 new). INFO: Got 0 tweets for Gipsy%20Kids%20Festival%20since%3A2018-09-07%20until%3A2018-09-11. INFO: Got 0 tweets (0 new). INFO: Got 0 tweets for Gipsy%20Kids%20Festival%20since%3A2018-07-26%20until%3A2018-07-30. INFO: Got 0 tweets (0 new).

pachocamacho1990 commented 6 years ago

Hi !! yes I had same problem yesterday, I am using latest version of Python 3.x running on Google Datalab. I tried different simple queries and is not working like it used to do !. I am starting to think if maybe it is due to a possible change in the twitter search-webpage or something !

johnjacobkenny commented 5 years ago

I am having a similar issue where only the latest date range receives the tweet data while older date ranges show INFO: Got 0 tweets (0 new). I'm thinking this may be due the the scraper not picking up the load more or next page to fetch the remaining. Could someone guide on what can be done regarding this. Thanks!

cdsimmons commented 5 years ago

Sorry, not sure if this is the right place... but I couldn't figure out how to resolve this myself and ended up writing my own node+selenium scraper to do what I needed - https://gist.github.com/cdsimmons/b676bf41edbf3d1e8ce3c11a5ded9b5a

johnjacobkenny commented 5 years ago

In response to my previous comment, I was able to make this work using a different network, so as far as I can see it still is working. Thanks!