twintproject / twint

An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.
MIT License
15.66k stars 2.72k forks source link

QUESTION: Search by User Id #629

Closed ghost closed 4 years ago

ghost commented 4 years ago

EDIT: Before you waste your time reading this, I eventually clue in that there is a configuration option of c.User_id. Doh!


Apologies if this has been discussed somewhere already - I searched through the issues and didn't see anything specific to this.

When I teach analysts about the use of Twitter for OSINT, one of the very first things I tell them is to never rely on the username to find accounts because users can (and bad guys often do!) deliberately change their usernames regularly if they are trying to hide from, well, people using Twitter for OSINT. I once watched an extremist account back in the bad old days (you know, 2014 or so) change his account name 30 times in 45 days. If you rely on the username to follow that guy, he disappears.

Since usernames are mutable, I have always used the user ID as the primary identifier for accounts. Is there any way to incorporate searching via twint using the user ID instead of the username?

ghost commented 4 years ago

I should say, I assume this is probably not possible since the Twitter advanced search system doesn't seem to allow it, but I thought I'd ask just in case.

For other analysts, I'd suggest that if you find that a username you were following doesn't seem to exist any more, go try a tool like https://tweeterid.com/ and put in the user ID of the account and see if it still exists with a new username.

ghost commented 4 years ago

For anyone wanting to try and incorporate an id lookup into their script before building the config for twint, you could do something like this:

import requests
from bs4 import BeautifulSoup

userid='12'
url = f'http://twittermoneybot.com/converter/?contype=2&q={userid}'
r = requests.get(url)
soup = BeautifulSoup(r.text, 'html.parser')
username = soup.find('input', {'id' : 'result'}).get('value')

c.twint.Config()
c.Username = username

Note I just chose one of the online Twitter userid -> username websites more or less at random based on the fact that its html was easy to parse. You could use a different site if you wanted, you'd just have to figure out what the appropriate query looks like and where the returned username is in the HTML.

NoelTautges commented 4 years ago

You can get a user's username from their ID through Twitter's intent system, though I am not sure whether it is rate limited or not.

E.g. https://twitter.com/intent/user?user_id=12 will always show Jack Dorsey's Twitter account details, no matter his username.

ghost commented 4 years ago

Thanks, that's really useful. I'm just coming back to looking at Twitter after a number of years doing other things, and there have clearly been some changes I have to catch up on! That's certainly easier than relying on a dodgy third party site.

Changing up my code then would look like this.

import requests
from bs4 import BeautifulSoup

userid='12'
url = f'https://twitter.com/intent/user?user_id={userid}'
r = requests.get(url)
soup = BeautifulSoup(r.text, 'html.parser')
username = soup.select_one("span[class=nickname]").text[1:] # to strip the '@'

c.twint.Config()
c.Username = username
ghost commented 4 years ago

Nevermind, now I feel dumb because I just realized there is literally a configuration option of c.User_id. I don't know how I overlooked that!

Nothing to see here, move along. /embarassed