JustAnotherArchivist / snscrape

A social networking service scraper in Python
GNU General Public License v3.0
4.51k stars 712 forks source link

Specify the number of tweets to the snscrape.modules.twitter moduler #589

Open aoamusat opened 2 years ago

aoamusat commented 2 years ago
from snscrape.modules import twitter

tweet_list = list(twitter.TwitterSearchScraper("Spacex").get_items())

The get_items() should be passed an argument n to specify the number of tweets to get from the TwitterSearchScraper

TheTechRobo commented 2 years ago

That's meant for you to do. For example:

tweet_generator = twitter.TwitterSearchScraper("Spacex").get_items()
tweet_list = []
max = 100
for count, tweet in tweet_generator.enumerate():
    if count >= max:
         break # reached max
    tweet_list.append(tweet)

List comprehension would probably work too but I'm at school.

JustAnotherArchivist commented 2 years ago

itertools.islice would be the most elegant approach. But yeah, this is best solved outside of snscrape since there are so many different ways how someone might want to filter results. First n results is probably a somewhat common one, but an itertools.islice example in the future documentation would be a better solution for that.