JustAnotherArchivist / snscrape

A social networking service scraper in Python
GNU General Public License v3.0
4.47k stars 709 forks source link

Specify the number of tweets to the snscrape.modules.twitter moduler #589

Open aoamusat opened 1 year ago

aoamusat commented 1 year ago
from snscrape.modules import twitter

tweet_list = list(twitter.TwitterSearchScraper("Spacex").get_items())

The get_items() should be passed an argument n to specify the number of tweets to get from the TwitterSearchScraper

TheTechRobo commented 1 year ago

That's meant for you to do. For example:

tweet_generator = twitter.TwitterSearchScraper("Spacex").get_items()
tweet_list = []
max = 100
for count, tweet in tweet_generator.enumerate():
    if count >= max:
         break # reached max
    tweet_list.append(tweet)

List comprehension would probably work too but I'm at school.

JustAnotherArchivist commented 1 year ago

itertools.islice would be the most elegant approach. But yeah, this is best solved outside of snscrape since there are so many different ways how someone might want to filter results. First n results is probably a somewhat common one, but an itertools.islice example in the future documentation would be a better solution for that.