taspinar / twitterscraper

Scrape Twitter for Tweets
MIT License
2.4k stars 578 forks source link

How can we also get amount of followers for the user whos tweet we scrape? #103

Closed eimis41 closed 6 years ago

eimis41 commented 6 years ago

Added:

followers = tweet.find('li', 'ProfileNav-item ProfileNav-item--followers').find( 'span','ProfileNav-value')['data-count']

to the tweet.py -> def from_soup(cls, tweet)

but getting error:

ERROR:root:An unknown error occurred! Returning tweets gathered so far. Traceback (most recent call last): File "....query.py", line 96, in query_tweets_once pos is None File "....query.py", line 39, in query_single_page return tweets, "TWEET-{}-{}".format(tweets[-1].id, tweets[0].id) IndexError: list index out of range INFO: Got 0 tweets (0 new).

Any ideas?

taspinar commented 6 years ago

I have implemented such a feature on the web-app version of twitterscraper. Can you have a look at it and see if that is what you need?

http://twitterscraper.nl/