JustAnotherArchivist / snscrape

A social networking service scraper in Python
GNU General Public License v3.0
4.49k stars 712 forks source link

reddit search using library #286

Closed NikinGeethila closed 3 years ago

NikinGeethila commented 3 years ago

Hi, could you please guide me on how to make a reddit search using the python library?

TheTechRobo commented 3 years ago

snscrape with no options returns this:

usage: snscrape [-h] [--version] [-v] [--dump-locals] [--retry N] [-n N] [-f FORMAT | --jsonl] [--with-entity] [--since DATETIME]
                [--progress]
                {vkontakte-user,weibo-user,telegram-channel,facebook-group,reddit-user,reddit-subreddit,reddit-search,twitter-search,twitter-tweet,instagram-user,instagram-hashtag,instagram-location,facebook-user,facebook-community,twitter-user,twitter-hashtag,twitter-list-posts,twitter-profile}
                ...
snscrape: error: the following arguments are required: scraper

Notice the vkontakte-user,weibo-user,[...]reddit-user,reddit-subreddit,reddit-search,[...]. Those are scrapers.

Try this. It will show you Reddit links pertaining to the search-term "Hello world".

snscrape reddit-search "Hello world"

You can search through specifically users and subreddits too, with their respective scrapers.

Hope this helps 😊

JustAnotherArchivist commented 3 years ago

@TheTechRobo covered the CLI. For the library usage, it's a bit tricky to understand what's going on in the Reddit module. Here's an example with the user scraper:

import snscrape.modules.reddit

scraper = snscrape.modules.reddit.RedditUserScraper('JustAnotherArchivist')
for i, item in enumerate(scraper.get_items()):
    print(item.json())
    if i >= 10:
        break

The subreddit and search scrapers are RedditSubredditScraper(subredditName) and RedditSearchScraper(searchTerm), respectively.

(Since this direct usage is currently undocumented, I cannot guarantee that it won't change without warning in backwards-incompatible ways in the future, although I have no immediate plans to do so.)

NikinGeethila commented 3 years ago

Thanks a lot. That was very helpful.