snscrape from with script

JustAnotherArchivist / snscrape

A social networking service scraper in Python

GNU General Public License v3.0

4.33k stars 699 forks source link

Closed p-dre closed 3 years ago

p-dre commented 3 years ago

Is there a way to use snscrape inside a script and not over the terminal?

If I try I get only the object. Or if that not possible how I could impliment a rotating proxy such that I can run the scraper over the cmd.

`import snscrape as sn
sn.modules.twitter.TwitterSearchScraper("corona lang=en")

<snscrape.modules.twitter.TwitterSearchScraper at 0x16f14ff45c8> `

JustAnotherArchivist commented 3 years ago

import snscrape.modules.twitter
for tweet in snscrape.modules.twitter.TwitterSearchScraper('corona lang:en').get_items():
    # do something

However, as this internal structure is currently undocumented, it may change at any time.

tiger-cpu commented 3 years ago

for using TwitterSearchScraper, do we need Twitter Premium Api Access? Does it search historical tweets? old than 30 days?