Scrape from profile - Githubissues

taspinar / twitterscraper

Scrape Twitter for Tweets

MIT License

2.41k stars 579 forks source link

Scrape from profile #140

Closed taspinar closed 6 years ago

taspinar commented 6 years ago

use a different INIT and RELOAD URL to scrape all tweets from an users profile page whenever the argument -u or --user is set. -update changelog / setup / README.rst

lapp0 commented 6 years ago

Nice work. Only issue I see is the docs are a bit confusing here:

You can also scraped all tweets written by retweetet by a specific user. This can be done by adding the ``-u / --user`` argument to the query, which should be the profile username.

"which should be the profile username" implies that you are setting the user with --user realDonaldTrump when in reality you're setting a boolean determining whether you're in profile mode.

lapp0 commented 6 years ago

You may also want to add https://developer.twitter.com/en/docs/tweets/search/guides/standard-operators.html to the notes

lapp0 commented 6 years ago

You also should remove this from the readme

TO DO
=====

-  Twitterscraper can not retrieve retweets.
-  Add caching potentially? Would be nice to be able to resume scraping
   if something goes wrong and have half of the data of a request cached
   or so.

For the 2nd part, I created an issue https://github.com/taspinar/twitterscraper/issues/141