search

JustAnotherArchivist / snscrape

A social networking service scraper in Python
GNU General Public License v3.0
4.39k stars 702 forks source link

Getting blank csv files after scraping #221

Closed vngoru closed 3 years ago

vngoru commented 3 years ago

Hey @JustAnotherArchivist I have tried using snscrape but code is working on coming to the csv files where I'm expecting the tweets I'm getting blanks.

Below is just a code snippet 

# Using OS library to call CLI commands in Python
os.system('snscrape --jsonl --max-results {} --since {} twitter-search "{} until:{}"> text-query-tweets.json'.format(tweet_count, since_date, text_query, until_date))

# Reads the json generated from the CLI command above and creates a pandas dataframe
tweets_df1 = pd.read_json('text-query-tweets.json', lines=True)

# Displays first 5 entries from dataframe
tweets_df1.head()

# Export dataframe into a CSV
tweets_df1.to_csv('text-query-tweets.csv', sep=',', index=False)
vngoru commented 3 years ago

@JustAnotherArchivist Hey why did you close the issue?

JustAnotherArchivist commented 3 years ago

Please read the pinned issue. I can't provide support for downstream use of snscrape output.