Cannot scrape by location, begin date, end date at the same time

[huseyn-guliyev]

Firstly, thanks for such a nice contribution, I find it very useful. Lucky I am that I found this library for my research.

The problem i have is that I cannot scrape tweets by location, begin date, end date at the same time.

I have scraper code for location like:

loc = '40.409264, 49.867092, 10km'
pd.DataFrame(itertools.islice(sntwitter.TwitterSearchScraper(
    'geocode:"{}"'.format(loc)).get_items(), 50))

and for begin and end date like that:

import os
# Setting variables to be used in format string command below
tweet_count = 500
text_query = "a"
since_date = "2020-06-01"
until_date = "2020-07-31"

# Using OS library to call CLI commands in Python
os.system('snscrape --jsonl --max-results {} --since {} twitter-search "{} until:{}"> text-query-tweets.json'.format(tweet_count, since_date, text_query, until_date))

and could not find any code that filter for all three category. It would be very helpful.

Note: I have looked to source code and could not find a way

Thanks in advance

[JustAnotherArchivist]

This query seems to work fine for me: geocode:40.409264,49.867092,10km since:2020-06-01 until:2020-08-01. CLI; snscrape --jsonl twitter-search 'geocode:40.409264,49.867092,10km since:2020-06-01 until:2020-08-01' API: snscrape.modules.twitter.TwitterSearchScraper('geocode:40.409264,49.867092,10km since:2020-06-01 until:2020-08-01') Your quoted version with spaces is also fine, just a bit more complicated than needed.

(By the way, note that the until filter is exclusive, i.e. the above fetches posts from June and July whereas your second command would miss posts from 31 July.)

[huseyn-guliyev]

Thanks a lot, dear @JustAnotherArchivist , CLI did not work, but API worked for me. Lack of documentation and examples made me to take your time, I am very sorry about it. Hope that I will end my research as I wished and cite your contribution too.

[JustAnotherArchivist]

No problem, happy to help. And yeah, I hope I get around to writing documentation sometime. Good luck with your research!