Mathieu2301 / TradingView-API

📈 Get real-time stocks from TradingView
1.31k stars 307 forks source link

`TradingView.loginUser` fails : 'Please confirm that you are not a robot by clicking the captcha box.' #202

Open NVentimiglia opened 1 year ago

NVentimiglia commented 1 year ago

Describe the bug TradingView.loginUser fails

Message:

'Please confirm that you are not a robot by clicking the captcha box.'

Stack :

'Error: Please confirm that you are not a robot by clicking the captcha box.\n at Object.loginUser (D:\Projects\StockDev\tradingview-dev\node_modules\@mathieuc\tradingview\src\miscRequests.js:376:27)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)'

To Reproduce Run the example with valid credentials. This failed the first time.

 TradingView.loginUser(login_name, login_code, true).then((u) => {
            user = u;
            console.log('User:', user);
            console.log('Sessionid:', user.session);
        }).catch((err) => {
            console.error('Login error:', err.message);
            client.end();
        });

Expected behavior loginUser works.

Environment:

Additional context Is there a way we can prompt the user (me) to view and solve the captcha? Is there a way I can get and apply my session Id from chrome ?

Mathieu2301 commented 1 year ago

Hello, the loginUser function can be used to generate a sessionid and a signature that must be stored somewhere. I don't know how long a session can be used, but I've never had any expiry problems. I'd say you're safe for at least 6 months / 1 year.

If the function fails the first time you call it, you should find your sessionid+signature pair in your browser cookies (sessionid and sessionid_sign).

You can manage and clean up your open TradingView sessions in 'ACCOUNT' > 'SETTINGS' > 'PRIVACY AND SECURITY' (https://fr.tradingview.com/u/{YOUR_USERNAME}/#security). In my experience, TradingView does not often require captcha verification. So if you clean up your sessions and wait a while, the loginUser function should work fine (but it doesn't normally need to be used more than once).

MakonDev commented 1 year ago

Noticing this today as well, I'll try to follow those steps, haven't had this issue before, reusing tokens usually works fine until now

NVentimiglia commented 1 year ago

You can manage and clean up your open TradingView sessions in 'ACCOUNT' > 'SETTINGS' > 'PRIVACY AND SECURITY' (https://fr.tradingview.com/u/{YOUR_USERNAME}/#security). In my experience, TradingView does not often require captcha verification. So if you clean up your sessions and wait a while, the loginUser function should work fine (but it doesn't normally need to be used more than once).

This has no effect.

If the function fails the first time you call it, you should find your sessionid+signature pair in your browser cookies (sessionid and sessionid_sign).

This worked! I was able to search for sessionid and sessionid_sign in chrome dev tools and pass them into the TradingView.getUser function successfully.

gasparesganga commented 10 months ago

This issue is starting to affect the library in a heavy way unfortunately, at least for people like me who is using it in a fully automated fashion from a 24/7 running bot.

Captcha solving requests have become more and more frequent, almost at a 100% rate of new session creation. At the same time, session expiry time was shortened significatively, going from a few months like it used to be, to a few days right now (that seems to be absolutely random though, ranging from a few hours to several weeks in my case).

Session cleanup in TV settings does not seem to help, leaving us with the only option of manually grab a new session ID/signature from browser cookies, thus defeating the purpose of an automted bot, especially when this happens at night.

Has anybody found a way to whitelist an IP/server in TradingView settings? If we could tell TV "look buddy, I trust this particular computer and/or IP, no captcha needed for it" we would be golden!

clins1994 commented 9 months ago

Captcha solving requests have become more and more frequent, almost at a 100% rate of new session creation. At the same time, session expiry time was shortened significatively, going from a few months like it used to be, to a few days right now (that seems to be absolutely random though, ranging from a few hours to several weeks in my case).

Interesting ... What does your sessionid token expiry say inside the cookie? Mine are saying 3 months. Could it be TradingView is doing some fancy rate limiting based on usage? Like if you send an inhuman amount of requests they start being more aggressive with the captchas. Probably not since I got captcha on my first attempt without even sending anything to them lol.

gasparesganga commented 9 months ago

@clins777 I had thought about that at first (I am doing about 4000/5000 connections a day from my server, long story but that's the most reliable and low latency way for my use case), but then it started to ask for a captcha even from my PC with a brand new IP.

I have to say that the session cleanup in TV seems to have taken effect a few hours after it was performed and it's been more than a week without TV asking for a captcha. Turns out Mathieu was right (again!) ;) It was just a matter of waiting a few hours after deleting all connections from TV settings from before attempting to connect from the same IP. It maybe just a coincidence, but we cannot prove it.

clins1994 commented 9 months ago

I have to say that the session cleanup in TV seems to have taken effect a few hours after it was performed and it's been more than a week without TV asking for a captcha.

Glad it worked out. Not glad we're still in the dark about how the captcha system works.

PS rant: TradingView should do one for the community and standardize the authentication

gasparesganga commented 9 months ago

@clins777 Ironically, tonight it went bad again and started to ask for a captcha, almost like it sensed we were kind of relieved that session cleanup worked :/

If only we could ask TV team for a kind of IP whitelist feature that would be great!

clins1994 commented 9 months ago

@gasparesganga that sucks. By the way how are you managing the session? Do you login every time you scrape or are you trying to re-use the session cookies from previous logins? I was trying to setup an automation using an account that was already logging in multiple times from a python server. So my hunch is that that account was already in the red for captchas and that's why my first login asked for captcha.

If only we could ask TV team for a kind of IP whitelist feature that would be great!

That would be great. But they probably don't want to give out the data for free so careful what you wish for 😢

gasparesganga commented 9 months ago

@clins777 My system (actually I didn't develop it, Mathieu did an incredible job building it around his library and I just added a couple of things here and there. It is totally worth having him develop something like that instead of trying to do it yourself if you are not 100% sure what you are doing with TV websocket) uses this library, hence it logs in using email/password only when the session cookie is rejected by TV server. Before this captcha thing, it used to work for many months with the same session! I believe the session expiry and captcha were just enforced in a more strictly/fequent fashion recently.

About the IP whitelisting, of course we should be careful asking for it. Maybe in the likes of I am not using your app but rather logging in and out every time from my browser that deletes all personal data and cookies when I close the tab, something like that. Also, I am using a Premium account. That could be a feature only available to high end plans (Plus & Premium) and they would be happy :)

daviddwlee84 commented 6 months ago

Is it possible to support 2FA? But I think it would be more complex than this issue.

node UserLogin.js username password
Login error: connect ETIMEDOUT 54.234.18.200:443
clins1994 commented 6 months ago

@NVentimiglia is this still happening to you?