search

minamotorin / twint

An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.
MIT License
46 stars 17 forks source link

followers: CRITICAL:root:twint.feed:Follow:IndexError #10

Open yashodhank opened 2 years ago

yashodhank commented 2 years ago

Initial Check

I am unable to get followers or following output

$ twint -u twitter_handle --followers
CRITICAL:root:twint.feed:Follow:IndexError
$ twint -u twitter_handle --following
CRITICAL:root:twint.feed:Follow:IndexError

My venv

git clone --depth=1 https://github.com/minamotorin/twint
pip3 install . -r requirements.txt
(venv) root@dev:~/twinproject/twint# python --version
Python 3.8.10
(venv) root@dev:~/twinproject/twint# pip3 --version
pip 20.0.2 from /root/twinproject/venv/lib/python3.8/site-packages/pip (python 3.8)

Thank you so much for sharing bug fixes and updates!

minamotorin commented 2 years ago

I know, but unfortunately, I think there is no way to scrape following or followers without a Twitter account now.

Even in the official client, you need to be logged in to see it. So, I suppose an QAuth Token is required to request following or followers data. In any case, I cannot fix this until I find a way to see it without logging in.

Actually, I tried the following API url with reference to gallery-dl's code (please replace <userId> by user ID).

https://twitter.com/i/api/graphql/mIwX8GogcobVlRwlgpHNYA/Following?variables=%7B%22userId%22%3A%22<userId>%22%2C%22count%22%3A100%2C%22includePromotedContent%22%3Afalse%2C%22withSuperFollowsUserFields%22%3Atrue%2C%22withBirdwatchPivots%22%3Afalse%2C%22withDownvotePerspective%22%3Afalse%2C%22withReactionsMetadata%22%3Afalse%2C%22withReactionsPerspective%22%3Afalse%2C%22withSuperFollowsTweetFields%22%3Atrue%2C%22withClientEventToken%22%3Afalse%2C%22withBirdwatchNotes%22%3Afalse%2C%22withVoice%22%3Atrue%2C%22withV2Timeline%22%3Afalse%2C%22%5F%5Ffs%5Finteractive%5Ftext%22%3Afalse%2C%22%5F%5Ffs%5Fdont%5Fmention%5Fme%5Fview%5Fapi%5Fenabled%22%3Afalse%7D

But the response said Authorization: Denied by access control: unspecified reason. gallery-dl https://twitter.com/<username>/following also doesn't work without logging in.

This is the reason why I cannot fix this.