Open flxai opened 3 years ago
Do you think it might be better to drop the parameter query_type
and make it implicit? So query_string
s that start with a #
are implicitly hashtags, so are cashtags with a $
at the beginning and everything else must be a user's account?
Made it implicit now. Think this to be a more intuitive user experience. It works like before now, but allows for #hashtag
or $cashtag
use like so:
import nitter_scraper
from nitter_scraper import NitterScraper
from pprint import pprint
queries = ["dgnsrekt", "#ToTheMoon", "$USDT"]
print("Scraping with local nitter docker instance.")
with NitterScraper(host="0.0.0.0", port=8008) as nitter:
for query in queries:
print('=' * 80, '\n', query, '\n', '=' * 80)
tweets = nitter.get_tweets(query, pages=1)
for tweet in tweets:
print('-' * 80)
pprint(tweet.dict())
print(tweet.json(indent=4))
Or with an arguable bit more readibility borrowing colored output:
import nitter_scraper
from nitter_scraper import NitterScraper
from pprint import pformat
from pygments import highlight
from pygments.lexers import PythonLexer
from pygments.formatters import TerminalFormatter
def pprint_color(obj):
print(highlight(pformat(obj), PythonLexer(), TerminalFormatter()))
queries = ["dgnsrekt", "#ToTheMoon", "$USDT"]
print("Scraping with local nitter docker instance.")
with NitterScraper(host="0.0.0.0", port=8008) as nitter:
for query in queries:
print('=' * 80, '\n', query, '\n', '=' * 80)
tweets = nitter.get_tweets(query, pages=1)
for tweet in tweets:
print('-' * 80)
pprint_color(tweet.dict())
This branch adds the ability to download tweets not only for a profile, but also for hashtags or cashtags.
Changes were made to the functions
get_tweets
andpagination_parser
innitter_scraper/tweets.py
andget_tweets
innitter_scraper/nitter.py
. Please tell me if you're okay with the implementation or have suggestions for improvement.Example usage for hashtags (leading
#
):Example for cashtags (leading
$
):