search

flightplan-tool / flightplan

Search for award inventory using Node.js and Headless Chrome!
Apache License 2.0
142 stars 41 forks source link

SQ Login TImeout #29

Closed ferni7 closed 5 years ago

ferni7 commented 5 years ago

Trying to search SQ (0.3.28) and getting timeouts on login, it never succeeds, is this just me or an issue?

andrew@fp:~$ flightplan search -w SQ -f MEL -t SIN -c first -s 2018-12-06 -e 2018-12-31 -q 2 -h --force flightplan search 0.3.28 Searching 26 days of award inventory: 12/06/2018 - 12/31/2018 [SQ] DEPARTURE [SIN -> MEL] - 2018-12-06 (2 Passengers) [SQ] info Logging in... [SQ] error Unexpected error occurred while searching! { TimeoutError: waiting for selector "#kfLoginPopup #membership-1" failed: timeout 30000ms exceeded at new WaitTask (/usr/local/share/.config/yarn/global/node_modules/puppeteer/lib/FrameManager.js:991:28) at Frame._waitForSelectorOrXPath (/usr/local/share/.config/yarn/global/node_modules/puppeteer/lib/FrameManager.js:882:12) at Frame.waitForSelector (/usr/local/share/.config/yarn/global/node_modules/puppeteer/lib/FrameManager.js:836:17) at Frame.waitFor (/usr/local/share/.config/yarn/global/node_modules/puppeteer/lib/FrameManager.js:821:19) at Page.waitFor (/usr/local/share/.config/yarn/global/node_modules/puppeteer/lib/Page.js:1022:29) at module.exports.login (/usr/local/share/.config/yarn/global/node_modules/flightplan-tool/src/engines/sq/searcher.js:34:18) at at process._tickCallback (internal/process/next_tick.js:189:7) name: 'TimeoutError' } [SQ] DEPARTURE [SIN -> MEL] - 2018-12-07 (2 Passengers)

jd20 commented 5 years ago

Working for me, when I tried just now. Can you try without -h, and verify visually if the login page is loading or not?

ferni7 commented 5 years ago

This is on a headless linux box so i can't run it without -h ...

jd20 commented 5 years ago

Ok, I'm seeing the same timeout error when I enable headless mode, for SQ only. It's likely that SQ has figured out a way to detect that you're using a headless browser (there's a few ways they can tell, though I've patched most of them), and is blocking you on the basis of that. I'll need to dig a bit deeper, to see what's going on.

ferni7 commented 5 years ago

Ok thanks for confirming :)

On Fri, 7 Dec 2018 at 13:45, Jason notifications@github.com wrote:

Ok, I'm seeing the same timeout error when I enable headless mode, for SQ only. It's likely that SQ has figured out a way to detect that you're using a headless browser (there's a few ways they can tell, though I've patched most of them), and is blocking you on the basis of that. I'll need to dig a bit deeper, to see what's going on.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/flightplan-tool/flightplan/issues/29#issuecomment-445104291, or mute the thread https://github.com/notifications/unsubscribe-auth/AOWZdSY2G2bLB8KRct_DYjWqbzc5JPG2ks5u2dZRgaJpZM4ZHlhO .

jd20 commented 5 years ago

This is now fixed, please upgrade to 0.3.29.

Basically, what happened is I added a fix in 0.3.28 improve the non-headless experience, but it messed up window metrics in headless mode, which caused SQ's website to think it was rendering on a mobile device. This caused the login link to disappear (because it's hidden in a menu on their mobile layout), which was causing the login to fail. The window metrics are back to normal now in 0.3.29.