Scraper Returns Error: {"errors":[{"code":366,"message":"Missing data."}]} when using the login() function

[aurmaza]

When using the login function with username, password and email on a local instance of scraper, it works and I am able to use functions that need the login function. Furthermore, isLoggedIn also returns true. However, when pushing the same code to a server on the cloud, I get the error below.

I am using the same exact inputs that I used on local on the cloud server This code is not run client side so I am not using the proxy but I tried anyway and it gave further errors on both local and the cloud server. Is there any configurations needed that I am unaware of? Im also using express.js

[witherBattler]

Same exact issue with my app

[witherBattler]

Hey, what hosting service do you use? Also, do you do app.enable("trust proxy") in your code or not?

[witherBattler]

I did some logging of the onboardingTaskUrl on line 266 of auth-user.js (as well as headers and data (body) that is send)

Here's what's being sent locally (with cleared cookies):

https://api.twitter.com/1.1/onboarding/task.json {} {"flow_name":"login","input_flow_data":{"flow_context":{"debug_overrides":{},"start_location":{"location":"splash_screen"}}}}
https://api.twitter.com/1.1/onboarding/task.json {} {"flow_token":"g;172191966613875304:-1721919666399:1sPWqs62lTIAjRE3SEn0wZ8L:0","subtask_inputs":[{"subtask_id":"LoginJsInstrumentationSubtask","js_instrumentation":{"response":"{}","link":"next_link"}}]}
https://api.twitter.com/1.1/onboarding/task.json {} {"flow_token":"g;172191966613875304:-1721919666399:1sPWqs62lTIAjRE3SEn0wZ8L:1","subtask_inputs":[{"subtask_id":"LoginEnterUserIdentifierSSO","settings_list":{"setting_responses":[{"key":"user_identifier","response_data":{"text_data":{"result":"allttt1807501"}}}],"link":"next_link"}}]}
https://api.twitter.com/1.1/onboarding/task.json {} {"flow_token":"g;172191966613875304:-1721919666399:1sPWqs62lTIAjRE3SEn0wZ8L:7","subtask_inputs":[{"subtask_id":"LoginEnterPassword","enter_password":{"password":"hopefullythismakemoney","link":"next_link"}}]} // this part is different from production 
https://api.twitter.com/1.1/onboarding/task.json {} {"flow_token":"g;172191966613875304:-1721919666399:1sPWqs62lTIAjRE3SEn0wZ8L:8","subtask_inputs":[{"subtask_id":"AccountDuplicationCheck","check_logged_in_account":{"link":"AccountDuplicationCheck_false"}}]}
https://api.twitter.com/1.1/onboarding/task.json {} {"flow_token":"g;172191966613875304:-1721919666399:1sPWqs62lTIAjRE3SEn0wZ8L:14","subtask_inputs":[]}

Here is what's being sent in production (render.com)

https://api.twitter.com/1.1/onboarding/task.json {} {"flow_name":"login","input_flow_data":{"flow_context":{"debug_overrides":{},"start_location":{"location":"splash_screen"}}}}
https://api.twitter.com/1.1/onboarding/task.json {} {"flow_token":"g;172191939981972814:-1721919400102:I3rJn2VppOBHfAaRmxNHT97d:0","subtask_inputs":[{"subtask_id":"LoginJsInstrumentationSubtask","js_instrumentation":{"response":"{}","link":"next_link"}}]}
https://api.twitter.com/1.1/onboarding/task.json {} {"flow_token":"g;172191939981972814:-1721919400102:I3rJn2VppOBHfAaRmxNHT97d:1","subtask_inputs":[{"subtask_id":"LoginEnterUserIdentifierSSO","settings_list":{"setting_responses":[{"key":"user_identifier","response_data":{"text_data":{"result":"allttt1807501"}}}],"link":"next_link"}}]}
https://api.twitter.com/1.1/onboarding/task.json {} 
{"flow_token":"g;172191939981972814:-1721919400102:I3rJn2VppOBHfAaRmxNHT97d:6","subtask_inputs":[{"subtask_id":"LoginEnterAlternateIdentifierSubtask","enter_text":{"link":"next_link"}}]} // different from when it runs locally

// and the error:
/opt/render/project/src/twitter-scraper/dist/auth-user.js:274
            return { status: 'error', err: new Error(await res.text()) };
                                           ^
Error: {"errors":[{"code":366,"message":"Missing data."}]}
    at TwitterUserAuth.executeFlowTask (/opt/render/project/src/twitter-scraper/dist/auth-user.js:274:44)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async TwitterUserAuth.handleEnterAlternateIdentifierSubtask (/opt/render/project/src/twitter-scraper/dist/auth-user.js:143:16)
    at async TwitterUserAuth.login (/opt/render/project/src/twitter-scraper/dist/auth-user.js:65:24)
    at async Scraper.login (/opt/render/project/src/twitter-scraper/dist/scraper.js:282:9)
    at async startTwitter (file:///opt/render/project/src/twitter.js:35:7)

So the beginning of the login process is the same, but not the end: locally, it does subtask_id LoginEnterPassword, while production subtask_id is LoginEnterAlternateIdentifierSubtask...

It seems like it tries to enter an alternate identifier subtask but it doesn't exist, hence why it says "missing data" (missing alternate identifier/alternate identifier subtask)

@karashiiro would it be possible to provide us with a quick fix for this?

[karashiiro]

Not sure what that subtask actually represents, it sounds like a 2FA challenge? I've been incredibly busy lately with some service outages in another project I maintain, but if there's any info on the flow floating around I can see what I can do.

[karashiiro]

Actually, that seems to have been added in this PR? https://github.com/the-convocation/twitter-scraper/issues/93

[witherBattler]

You were right, all I had to do was add the "email" parameter (used a phone number instead, though) I was afraid it would be required to also obtain the verification code, but for some reason there was no need.