search

bocchilorenzo / ntscraper

Scrape from Twitter using Nitter instances
MIT License
178 stars 29 forks source link

Is possible to stablish the country in wich you want scrape the tweets? #31

Closed AritzUMA closed 1 year ago

AritzUMA commented 1 year ago

I want to know if there is a possibility to know the location of the each tweet.

thanks

bocchilorenzo commented 1 year ago

The new version contains the filter for the location (and more). Simply add near="location" in the get_tweets call to restrict the locations of the tweets.

AritzUMA commented 1 year ago

I upgrade the library but now my code is not workingg properly with number of tweets

tweets = scraper.get_tweets(terms = termino,mode='term',number = 100,since= formatted_date_since, instance="https://nitter.ktachibana.party")

Fetching data: 0%| | 0/50 [00:00<?, ?user/s]First Attempt for termino: #Worlds2023 19-Nov-23 13:23:07 - Current stats for Worlds2023: 12 tweets, 0 threads... 19-Nov-23 13:23:13 - Current stats for Worlds2023: 27 tweets, 0 threads... 19-Nov-23 13:23:18 - Current stats for Worlds2023: 39 tweets, 0 threads... 19-Nov-23 13:23:23 - Current stats for Worlds2023: 49 tweets, 0 threads... 19-Nov-23 13:23:28 - Current stats for Worlds2023: 60 tweets, 0 threads... 19-Nov-23 13:23:32 - Current stats for Worlds2023: 73 tweets, 0 threads... 19-Nov-23 13:23:37 - Current stats for Worlds2023: 83 tweets, 0 threads... 19-Nov-23 13:23:41 - Current stats for Worlds2023: 95 tweets, 0 threads... 19-Nov-23 13:23:46 - Current stats for Worlds2023: 107 tweets, 0 threads... 19-Nov-23 13:23:51 - Current stats for Worlds2023: 116 tweets, 0 threads... 19-Nov-23 13:23:56 - Current stats for Worlds2023: 124 tweets, 0 threads... 19-Nov-23 13:24:00 - Current stats for Worlds2023: 130 tweets, 0 threads... 19-Nov-23 13:24:06 - Current stats for Worlds2023: 138 tweets, 0 threads... 19-Nov-23 13:24:10 - Current stats for Worlds2023: 145 tweets, 0 threads... 19-Nov-23 13:24:14 - Current stats for Worlds2023: 154 tweets, 0 threads... 19-Nov-23 13:24:19 - Current stats for Worlds2023: 165 tweets, 0 threads... 19-Nov-23 13:24:24 - Current stats for Worlds2023: 171 tweets, 0 threads... 19-Nov-23 13:24:29 - Current stats for Worlds2023: 179 tweets, 0 threads... 19-Nov-23 13:24:33 - Current stats for Worlds2023: 188 tweets, 0 threads... 19-Nov-23 13:24:38 - Current stats for Worlds2023: 192 tweets, 0 threads... 19-Nov-23 13:24:43 - Current stats for Worlds2023: 199 tweets, 0 threads... 19-Nov-23 13:24:47 - Current stats for Worlds2023: 208 tweets, 0 threads...

bocchilorenzo commented 1 year ago

Fixed, the intended behavior was that if "since" was set, then the "number" was bypassed. Now it always respects the number, and to scrape all the available tweets you simply don't set the "number" parameter.

AritzUMA commented 1 year ago

Now works, but it doesn't scrape te number of tweets stablished, it never arrives to 100, random low numbers of scraped tweets