search

flightplan-tool / flightplan

Search for award inventory using Node.js and Headless Chrome!
Apache License 2.0
142 stars 41 forks source link

BA Captcha timeout #11

Open dday4thedeceiver opened 5 years ago

dday4thedeceiver commented 5 years ago

So presumably at 45 queries or so, BA is now prompting with a captcha. Not sure yet if it appears again, will update. I haven't tried to see if multiple sub-45 day searches trigger this as well.

Is the cool-down period adjustable? Is that time or number of queries based? If so I can play with it and see what works.

As it is, the immediate problem is I don't always get a chance to answer the captcha as the search refreshes with the next query just before I'm about to hit submit. If the cool-down isn't the answer, then the parser needs to detect the captcha and wait much longer.

jd20 commented 5 years ago

Yes, throttling settings are set on config.throttling in src/ba/index.js. You can either choose a different profile (such as profiles.slow), or create a new profile in index/consts.js. The settings are:

delayBetweenRequests: Interval to enforce between successive requests (the interval is chosen randomly in the range provided after each request) requestsPerHour: Like it says, limits how many requests per hour can be run (but by inserting one big sleep, instead of small sleeps between each request). restPeriod: How often to enforce requestsPerHour. Smaller values will mean more frequent small sleeps, whereas bigger values will mean fewer bigger sleeps.

Cookies get cleared each time you run the search command, so my guess is that if you were to change your IP address and account and re-run the search command, the captcha would go away. I have no idea if they flag your IP or account (or both), if you figure out let us know :)

dday4thedeceiver commented 5 years ago

Thank you. New info so far: default cooldown didn't help, after 20 more requests I got the cooldown, and upon resuming got the captcha immediately. I'll play with the throttling settings and also account change and update.

jd20 commented 5 years ago

Btw, I've added some logic to the BA searcher, so that if it detects the captcha was presented, it will stop and wait for the user to solve it (as opposed to before, it would get confused and error out). You'll see a message on the console, prompting you to solve the captcha. Once the captcha is solved, searching will continue (the searcher checks every 5-10 seconds or so, to see if the captcha has been solved successfully).

Of course, this means you cannot run BA in headless mode (otherwise you wouldn't be able to interact with the browser to solve the captcha), but seems a reasonable trade off for now.

dday4thedeceiver commented 5 years ago

Beautiful! Haven't had time recently but I'm planning to really dig into it next weekend.

On Sat, Nov 10, 2018 at 2:54 AM Jason notifications@github.com wrote:

Btw, I've added some logic to the BA searcher, so that if it detects the captcha was presented, it will stop and wait for the user to solve it (as opposed to before, it would get confused and error out). You'll see a message on the console, prompting you to solve the captcha. Once the captcha is solved, searching will continue (the searcher checks every 5-10 seconds or so, to see if the captcha has been solved successfully).

Of course, this means you cannot run BA in headless mode (otherwise you wouldn't be able to interact with the browser to solve the captcha), but seems a reasonable trade off for now.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/flightplan-tool/flightplan/issues/11#issuecomment-437569071, or mute the thread https://github.com/notifications/unsubscribe-auth/ARDXzpLuxPt56KHPFS5XUAcNsSINX1crks5utpQngaJpZM4XWj7G .