JustAnotherArchivist / snscrape

A social networking service scraper in Python
GNU General Public License v3.0
4.31k stars 698 forks source link

can I append multiple account tweets into a single file? #100

Closed sameepdesai closed 3 years ago

sameepdesai commented 3 years ago
        for i in list:
                subprocess.call("snscrape --jsonl --max-results 1000 --since "+yes+" twitter-user " + i + " >temp.jsonl",shell=True)
JustAnotherArchivist commented 3 years ago

> is just a shell redirection. snscrape doesn't even know about the file at all. You can append to a file using >>.

gxwyz commented 3 years ago

if i use this command,it time since and until no Effective subprocess.call("snscrape --jsonl twitter-search 'from:" + line + " since 2020-09-01 until 2020-10-08' >> " + line + ".jsonl",shell=True)

gxwyz commented 3 years ago

if i use this:subprocess.call("snscrape --jsonl --max-results 100000 --since 2020-09-01 twitter-user " + line + " >> temp.jsonl",shell=True) then i only write the last line‘s data into the line.jsonl

JustAnotherArchivist commented 3 years ago

Works as expected for me:

import subprocess
subprocess.call("snscrape --jsonl -n 10 twitter-user textfiles >>test.jsonl", shell = True)
subprocess.call("snscrape --jsonl -n 10 twitter-user internetarchive >>test.jsonl", shell = True)

Produces a file with the ten most recent tweets by Jason followed by the ten most recent from IA.