"Cannot find twitter account with name = ", This happened suddenly today, but it was fine yesterday.

[Lxhnnn]

Issue Template

Please use this template!

Initial Check

If the issue is a request please specify that it is a request in the title (Example: [REQUEST] more features). If this is a question regarding 'twint' please specify that it's a question in the title (Example: [QUESTION] What is x?). Please only submit issues related to 'twint'. Thanks.

Make sure you've checked the following:

• [] Python version is 3.6;
• [] Updated Twint with pip3 install --user --upgrade -e git+https://github.com/twintproject/twint.git@origin/master#egg=twint;
• [] I have searched the issues and there are no duplicates of this issue/question/request.

Command Ran

Please provide the exact command ran including the username/search/code so I may reproduce the issue.

Description of Issue

Please use as much detail as possible.

Environment Details

Using Windows, Linux? What OS version? Running this in Anaconda? Jupyter Notebook? Terminal?

[THEDARKK]

Same

[Legnus]

Same here

[andywalden]

This issue is due to Twitter disabling the legacy interface and redirecting to the javascript page. At a minimum, anything scraping twitter without an API now must at least click the "Yes" button on the direct page. This change has broken this app entirely.

[mikaelyh]

Even though the disabling of the legacy interface, it was working until yesterday. Something new has broken this app.

[javad94]

Even though the disabling of the legacy interface, it was working until yesterday. Something new has broken this app.

Maybe because they launched their new API.

[skywd]

Even though the disabling of the legacy interface, it was working until yesterday. Something new has broken this app.

Maybe because they launched their new API.

Seems as though they have completely phased out requesting that page with XML. Request returns 400 error for me now.

[andywalden]

Yes, they disabled the ability to access the "legacy site" without a javascript "yes", clickthough. And then even the legacy site is sorely lacking granularity. You don't get a real timestamp, the pics, videos and other media aren't parsed out and threads are presented differently. It's a very primitive set of data to work from,

I don't actually use twint as I found it to be too restrictive for what I was trying to do, but obviously I'm watching the team to see how they react to this change. I maintain my own twitter bot and have converted the mobile.twitter.com experience to my objects, albeit, with reduced information.

[neuraleye]

Has the ability to crawl likes without authentication been completely nuked, or is it just a matter of passing the javascript clickthrough?

[andywalden]

It does not appear that likes, nor any stateful info, is available on the minimized mobile interface.

Puppeteer may be the next frontier: https://github.com/puppeteer/puppeteer#faq

[neuraleye]

Thank you!

Do you think twitter would lock accounts for suspicious behavior like scrolling 1K different people's likes in an hour?

[medhax]

Same issue here, in my case is also giving CRITICAL....: 'NoneType' object is not suscriptable.

Hope to get it restored soon :)

[rossrochford]

@andywalden I'm writing currently a Twitter scraper using Selenium with a custom asyncio wrapper. It's working well but getting a users' followers requires logging in. Happy to open source my progress and collaborate.

[yunusemrecatalcam]

I see that twitter legacy theme is not closed yet for favorites/followings/search pages. This means we only lost the profile page for now. Since other functions need user id in order to fetch the data, we still need to able to fetch profile page. I am trying to figure out a way to fetch profile page data without login or using webdriver. For those who interested, when you open a profile from browser; It uses a guest token named 'x-guest-token' to query an endpoint ('https://api.twitter.com/graphql/-xfUfZsnR_zqjFd-IfrN5A/UserByScreenName?variables=%7B%22screen_name%22%3A%22LeagueOfLegends%22%2C%22withHighlightedLabel%22%3Atrue%7D') that returns user data as json. firstly I have checked the timeout for that token, unluckily its died in a few hours. Now I am trying to get the token before every profile request.
Anyone working on that, I'd be really happy to talk about that and collaborate. Here is a discord link if you wanna closely communicate and rescue the twint together! We'll hack it out again! https://discord.gg/ze4Zjj

[darvell]

@yunusemrecatalcam Your discord link is expired but this shouldn't be much of a project. I haven't documented this properly so it's vague but a config endpoint is hit even if you're not logged in, you get an auth token among other things so you're "logged in" and without that, things break.

[pielco11]

twint -s "from:username" replaces twint -u username, works as expected

[THEDARKK]

But to make the --following command work, it's no longer possible.

sudo twint -u username --following -o test.txt

[pielco11]

You are right. But consider that the --following/--followers is intended to be more a PoC rather than else, honestly idk why people don't use the official API which allows you to get 15k followers/following in a couple of seconds

Usually I don't suggest to run twint as sudo, better install in the user's PATH

[subokita]

Similar issue here, also using "from:username" doesn't work on all users, even using Twitter's own search field.

[darvell]

Maybe we just need to modify twint to require user/pass just for the tokens?

[pielco11]

@subokita the from parameter is the one used by Twitter, so it's expected to work. In my test it works well so idk what you are getting

[subokita]

@pielco11 Yes, I used that "from" params for search very regularly for the past year, but I found out that, at least on my case, it doesn't work 100%, it won't surface any tweets from some valid active users, e.g. for my case "from:yellowkenworth" returns nothing, while yellowkenworth is an active user, not suspended or anything.

[pielco11]

I guess that your target has been shadow-banned, so his/her tweets do not compare in search results

[DiabetesLongTorsoMan]

@pielco11 I'm new to this library and you seem to know what's going on. Is this kind of issue typical where it causes much of the library to no longer work? And the issue isn't resolved for a while? I'm basically just wondering how long it usually takes for stuff like this to be resolved, and if it's likely that this library will stay broken due to the changes twitter made. Thank you!!!

[darvell]

I wonder if a logged in user would work. I've tried making sure all my requests have all the standard twitter header/cookie stuff that I'm logged in with....but nothing.

[yunusemrecatalcam]

You are right. But consider that the --following/--followers is intended to be more a PoC rather than else, honestly idk why people don't use the official API which allows you to get 15k followers/following in a couple of seconds

Usually I don't suggest to run twint as sudo, better install in the user's PATH

@pielco11 twitter api is limited when you want to scrape users on a scale. I have created a pull request that makes it possible to fetch favorite and followings again, lets check it out! https://github.com/twintproject/twint/pull/870

[shadow-samurai]

please help --following/--followers command in twint dont run and show error ValueError: Cannot find twitter account with name =XXXX I checked with 10 IP and dont run twitter api limit

[pielco11]

@yunusemrecatalcam Merged

@DiabetesLongTorsoMan tweets of shadowbanned users are not found not due to twint fail

[somepasta]

After updating twint from the latest commit I tried getting a list of followers/following using the --followers/--following commands. However, the error CRITICAL:root:twint.feed:Follow:IndexError shows up every single time. The full command I'm trying to use is twint -s "from:username" --following --user-full -o file.txt

[yunusemrecatalcam]

Follower page is not stable for now. I'm gonna check it

[pielco11]

@somepasta the -s argument is used just for the searches, for that trick does not work for followers/following

[somepasta]

@yunusemrecatalcam Thank you.

@pielco11 I see. I thought replacing with -s "from:username" could possibly solve this problem. Using -u username will always return in ValueError: Cannot find twitter account with name = username as mentioned above. Hoping there's a new way to scrape a list followings/followers soon, it was extremely useful and simple to use.

[shadow-samurai]

i want use from twint for get followers/following please help me twint show error : CRITICAL:root:twint.feed:Follow:IndexError

[pielco11]

Lookup function works partially (some fields can't be scraped)

import twint

c = twint.Config()
c.Username = 'NaturalIntelli4'
c.Store_object = True
c.Store_object_users_list = []
c.User_full = True
twint.run.Lookup(c)
user_info = c.Store_object_users_list[0]

Followers/following working if c.User_full = False and the user ID is specified

c = twint.Config()
c.User_id = "2550600458"
c.Username = "noneprivacy"
c.Limit = 20
twint.run.Following(c)

Please update with the pip+git Thanks to @yunusemrecatalcam for fixing

[shadow-samurai]

--user-full was very good switch in twint

[chevalier-thib]

Hello !

I currently have the same problem but with the Search function. The strange thing is that it works fine in command line. I am using the latest git version (installed today with the pip+git command).

My usage :

import twint

c = twint.Config()
c.Username = 'threatpunter'
c.Limit = 20
twint.run.Search(c)

results in ValueError: Cannot find twitter account with name = threatpunter

However in CLI :

$twint -s threatpunter --limit 20 Everything works fine and I get my 20 tweets.

I noticed that it works if you add in the user id in the config, however I do not have access to it and since it seems possible to get the tweets without it in the CLI I'm guessing it should not be necessary.

Thank you !

[yunusemrecatalcam]

Add a random string as user id and it'll work!

import twint
c = twint.Config()
c.Username = 'threatpunter'
c.User_id = 'randomstring'  # like this 
c.Limit = 20
twint.run.Search(c)

[chevalier-thib]

Thanks, it works !

[glazou]

To fetch tweet of a given user, the configuration that works for me is:

twint_config = twint.Config()
twint_config.Search = f"from:{username}"
twint.run.Search(twint_config)

[alexgeo20]

@pielco11 thank you very much for your help with this. I tried it with twint -s "from: username" as you mentioned and it worked!

[mikaelyh]

violation of antitrust laws

https://www.reuters.com/article/us-microsoft-linkedin-ruling-idUSKCN1AU2BV

[coltranetorres]

just encountered "Cannot find twitter account with name =" again just today, i've already upgraded to the latest version of twint, Search() does not work but Lookup() does work, please help me resolve this:(

tried running in jupyter notebook and command line, but still does not work ..