Create Twitter Scrapper

neokd / DataStorehouse

DataStoreHouse is an open-source project that aims to create a collaborative platform for gathering and sharing a wide variety of datasets. It provides a centralised repository where individuals and organisations can contribute, discover, and collaborate on diverse datasets for various domains.

https://datash.vercel.app

MIT License

18 stars 22 forks source link

Create Twitter Scrapper #86

Closed neokd closed 1 year ago

neokd commented 1 year ago

Create a class Twitter or X in scrapper folder to scrape twitter

Bchass commented 1 year ago

Few things to think about:

Do we want to use login creds?
If we want to bypass login creds, we'll need an API like: https://scrapfly.io but depends on how many calls we want a month
User profiles or random tweets?

neokd commented 1 year ago

Yes, I'm thinking of using two approaches one is with credentials that would be easy but since twitter has a rate limit we can also implement 3rd party APIs.