Getting error 429 during fare checks and logins

[StevenMassaro]

Version

Auto-Southwest Check-In v7.1

Browser Version

Docker image

Description

Recently, I've started getting these 429 errors when checking the prices for upcoming flights. I am not sure if they are truly a 429 error, caused by actually sending too many requests, or a different error being reported as a 429.

To Reproduce

1. Add flights to the app
2. Check for price differences

Expected Behavior

No 429 error

Relevant logs and program output

2024-01-09 13:54:00 DEBUG Process-3[fare_checker:30]: Checking current price for flight
2024-01-09 13:54:00 DEBUG Process-3[fare_checker:88]: Fetching reservation information
2024-01-09 13:54:02 DEBUG Process-3[utils:34]: Successfully made request after 1 attempts
2024-01-09 13:54:02 DEBUG Process-3[fare_checker:103]: Retrieving search information for the current flight
2024-01-09 13:54:03 DEBUG Process-3[utils:34]: Successfully made request after 1 attempts
2024-01-09 13:54:03 DEBUG Process-3[fare_checker:82]: Retrieving matching flights
2024-01-09 13:54:08 DEBUG Process-3[utils:41]: Failed to make request after 7 attempts: Too Many Requests 429
2024-01-09 13:54:08 DEBUG Process-3[utils:44]: Response body: {
    "code": 429999999,

    "message": "Error.",

    "messageKey": "ERROR",

    "httpStatusCode": "BAD_REQUEST",

    "requestId": "",

    "infoList": []
   }
2024-01-09 13:54:08 ERROR Process-3[reservation_monitor:102]: Requesting error during fare check. Too Many Requests 429. Skipping...

Additional context

No response

[jdholtz]

An update to my attempts to fix/identify this issue:

I can only reproduce this issue in Docker as of right now, which is really strange as it makes me think that Southwest has not actually improved their bot detection but rather something is happening within Docker that is causing it to be detected.

Within Docker, using Google Chrome instead of Chromium results in successfully logging in (Chromium is now failing to log in as of yesterday). However, Google Chrome still fails with a 429 during check-in/fare check.

[hildebrau]

I wonder if the issue is somehow Unix permission based. It feels like docker has been clamping down on assumed permissions recently. In another similar project I was toying with, I had to add several sudo wrappers around things. Maybe the header generation part is silently failing to create a temp file? Also, have you tried regenerating the headers after detecting a 429?

[wpkato]

I can only reproduce this issue in Docker as of right now, which is really strange as it makes me think that Southwest has not actually improved their bot detection but rather something is happening within Docker that is causing it to be detected.

I can't get develop or 7.1 branches to work in console, outside of docker. Just started happening yesterday. Had no problem last weekend when I had to check in for my flight (using 7.1 at the time, in console, outside of docker)

[jdholtz]

I can't get develop or 7.1 branches to work in console, outside of docker. Just started happening yesterday. Had no problem last weekend when I had to check in for my flight (using 7.1 at the time, in console, outside of docker)

Does the script fail to log in or do you just get 429s when fare checking?

[wpkato]

Seems to be failing, maybe 429 when just retrieving reservations to schedule

2024-01-27 18:56:49 DEBUG MainProcess[log:23]: Initialized the application 2024-01-27 18:56:49 DEBUG MainProcess[main:112]: Auto-Southwest Check-In v7.1 2024-01-27 18:56:49 DEBUG MainProcess[main:113]: Called with 2 arguments 2024-01-27 18:56:49 DEBUG MainProcess[config:107]: Initializing configuration file 2024-01-27 18:56:49 DEBUG MainProcess[config:135]: Reading the configuration file 2024-01-27 18:56:49 DEBUG MainProcess[config:139]: No configuration file found. Using defaults 2024-01-27 18:56:49 DEBUG MainProcess[config:118]: Creating configurations for 1 accounts 2024-01-27 18:56:49 DEBUG MainProcess[main:129]: Account added through CLI arguments 2024-01-27 18:56:49 DEBUG MainProcess[main:142]: Monitoring 1 accounts and 0 reservations 2024-01-27 18:56:49 DEBUG Process-1[reservation_monitor:154]: Acquiring lock... 2024-01-27 18:56:49 DEBUG Process-1[reservation_monitor:156]: Lock acquired 2024-01-27 18:56:49 DEBUG Process-1[reservation_monitor:178]: Retrieving reservations for account 2024-01-27 18:56:49 DEBUG Process-1[webdriver:103]: Starting webdriver for current session 2024-01-27 18:56:51 DEBUG Process-1[webdriver:119]: Using browser version: 120.0.6099.224 2024-01-27 18:56:51 DEBUG Process-1[webdriver:123]: Loading Southwest Check-In page 2024-01-27 18:56:54 DEBUG Process-1[webdriver:80]: Logging into account to get a list of reservations and valid headers 2024-01-27 18:56:55 DEBUG Process-1[webdriver:152]: Waiting for headers_set to be set 2024-01-27 18:56:55 DEBUG Process-1[webdriver:156]: headers_set set successfully 2024-01-27 18:56:56 DEBUG Process-1[webdriver:152]: Waiting for login_request_id to be set 2024-01-27 18:56:56 DEBUG Process-1[webdriver:144]: Login response has been received 2024-01-27 18:56:56 DEBUG Process-1[webdriver:156]: login_request_id set successfully 2024-01-27 18:56:56 DEBUG Process-1[webdriver:212]: Logging in failed for an unknown reason 2024-01-27 18:56:56 WARNING Process-1[reservation_monitor:187]: Encountered a Too Many Requests error while logging in. Skipping reservation retrieval 2024-01-27 18:56:56 DEBUG Process-1[reservation_monitor:167]: Lock released 2024-01-27 18:56:56 DEBUG Process-1[reservation_monitor:116]: Sleeping for 86393 seconds

[agoldh20]

if it makes a difference, I was using the app from docker:latest, and I was getting Requesting error during fare check. Too Many Requests 429. Skipping... error even though I wasn't checking fares, only checking in, and then it wouldn't check me in bc of the too many requests from fare checking

[mickgiles]

I had another 429 error and failure to check in only from docker, not to pile it on.

[agoldh20]

I also got another 429 on docker v5.1 this morning. Good thing I was up early enough to checkin only a minute late 😄

[jdholtz]

I've been working on trying to find the issue. Still haven't made much progress though. The logging in works much more reliably with Chrome rather than Chromium but it still hits a 429 when fare checking/checking in.

[evanchen7]

I'm on the develop branch and installed google-chrome-stable instead and set the browser_path to /usr/bin/google-chrome-stable. Was able to login, but the checking in process seems to be hanging.

I also updated the seleniumbase library to 4.23.2 (pretty sure this didn't do anything.

[jdholtz]

but the checking in process seems to be hanging.

Where does it hang? Can you send the logs? Also, is this in Docker or just Python?

[evanchen7]

Hangs after the last line, this is just python. Was previously running in docker.

2024-02-01 21:31:11 DEBUG MainProcess[main:112]: Auto-Southwest Check-In v7.1
2024-02-01 21:31:11 DEBUG MainProcess[main:113]: Called with 0 arguments
2024-02-01 21:31:11 DEBUG MainProcess[config:107]: Initializing configuration file
2024-02-01 21:31:11 DEBUG MainProcess[config:135]: Reading the configuration file
2024-02-01 21:31:11 DEBUG MainProcess[config:53]: Setting check fares to False
2024-02-01 21:31:11 DEBUG MainProcess[config:67]: Setting notification level to <NotificationLevel.INFO: 1>
2024-02-01 21:31:11 DEBUG MainProcess[config:80]: Using 1 notification services
2024-02-01 21:31:11 DEBUG MainProcess[config:84]: Setting retrieval interval to 24 hours
2024-02-01 21:31:11 DEBUG MainProcess[config:152]: Setting custom Browser path
2024-02-01 21:31:11 DEBUG MainProcess[config:118]: Creating configurations for 6 accounts
2024-02-01 21:31:11 DEBUG MainProcess[config:125]: Creating configurations for 0 reservations
2024-02-01 21:31:11 DEBUG MainProcess[main:142]: Monitoring 6 accounts and 0 reservations
2024-02-01 21:31:11 DEBUG Process-1[reservation_monitor:154]: Acquiring lock...
2024-02-01 21:31:11 DEBUG Process-1[reservation_monitor:156]: Lock acquired
2024-02-01 21:31:11 DEBUG Process-1[reservation_monitor:178]: Retrieving reservations for account
2024-02-01 21:31:11 DEBUG Process-1[webdriver:103]: Starting webdriver for current session
2024-02-01 21:31:11 DEBUG Process-2[reservation_monitor:154]: Acquiring lock...
2024-02-01 21:31:11 DEBUG Process-3[reservation_monitor:154]: Acquiring lock...
2024-02-01 21:31:11 DEBUG Process-4[reservation_monitor:154]: Acquiring lock...
2024-02-01 21:31:11 DEBUG Process-5[reservation_monitor:154]: Acquiring lock...
2024-02-01 21:31:11 DEBUG Process-6[reservation_monitor:154]: Acquiring lock...
2024-02-01 21:31:12 DEBUG Process-1[webdriver:119]: Using browser version: 121.0.6167.139
2024-02-01 21:31:12 DEBUG Process-1[webdriver:123]: Loading Southwest Check-In page
2024-02-01 21:31:14 DEBUG Process-1[webdriver:80]: Logging into account to get a list of reservations and valid headers
2024-02-01 21:31:15 DEBUG Process-1[webdriver:152]: Waiting for headers_set to be set
2024-02-01 21:31:15 DEBUG Process-1[webdriver:156]: headers_set set successfully
2024-02-01 21:31:17 DEBUG Process-1[webdriver:144]: Login response has been received
2024-02-01 21:31:17 DEBUG Process-1[webdriver:152]: Waiting for login_request_id to be set
2024-02-01 21:31:17 DEBUG Process-1[webdriver:156]: login_request_id set successfully
2024-02-01 21:31:17 DEBUG Process-1[webdriver:230]: First time logging in. Setting account name
Successfully logged in to REDACTED's account

2024-02-01 21:31:17 DEBUG Process-1[webdriver:152]: Waiting for trips_request_id to be set
2024-02-01 21:31:18 DEBUG Process-1[webdriver:148]: Upcoming trips response has been received
2024-02-01 21:31:18 DEBUG Process-1[webdriver:156]: trips_request_id set successfully
2024-02-01 21:31:18 DEBUG Process-1[reservation_monitor:197]: Successfully retrieved 0 reservations
2024-02-01 21:31:18 DEBUG Process-1[reservation_monitor:84]: Scheduling flight check-ins for 0 reservations
2024-02-01 21:31:18 DEBUG Process-1[checkin_scheduler:43]: 0 total flights were found
2024-02-01 21:31:18 DEBUG Process-1[checkin_scheduler:100]: 0 new flights found
2024-02-01 21:31:18 DEBUG Process-1[checkin_scheduler:104]: Scheduling 0 flights for check-in
2024-02-01 21:31:18 DEBUG Process-1[checkin_scheduler:116]: 0 flights are currently scheduled. Removing old flights
2024-02-01 21:31:18 DEBUG Process-1[checkin_scheduler:132]: Successfully removed old flights. 0 flights are now scheduled
2024-02-01 21:31:18 DEBUG Process-1[reservation_monitor:167]: Lock released
2024-02-01 21:31:18 DEBUG Process-1[reservation_monitor:116]: Sleeping for 86393 seconds
2024-02-01 21:31:18 DEBUG Process-2[reservation_monitor:156]: Lock acquired
2024-02-01 21:31:18 DEBUG Process-2[reservation_monitor:178]: Retrieving reservations for account
2024-02-01 21:31:18 DEBUG Process-2[webdriver:103]: Starting webdriver for current session
2024-02-01 21:31:18 DEBUG Process-2[webdriver:119]: Using browser version: 121.0.6167.139
2024-02-01 21:31:18 DEBUG Process-2[webdriver:123]: Loading Southwest Check-In page
2024-02-01 21:31:21 DEBUG Process-2[webdriver:80]: Logging into account to get a list of reservations and valid headers
2024-02-01 21:31:22 DEBUG Process-2[webdriver:152]: Waiting for headers_set to be set
2024-02-01 21:31:22 DEBUG Process-2[webdriver:156]: headers_set set successfully
2024-02-01 21:31:23 DEBUG Process-2[webdriver:144]: Login response has been received
2024-02-01 21:31:29 DEBUG Process-2[webdriver:190]: Login form failed to submit. Clicking login button again
2024-02-01 21:31:29 DEBUG Process-2[webdriver:152]: Waiting for login_request_id to be set
2024-02-01 21:31:29 DEBUG Process-2[webdriver:156]: login_request_id set successfully
2024-02-01 21:31:29 DEBUG Process-2[webdriver:230]: First time logging in. Setting account name
Successfully logged in toREDACTED's account

2024-02-01 21:31:29 DEBUG Process-2[webdriver:152]: Waiting for trips_request_id to be set
2024-02-01 21:31:30 DEBUG Process-2[webdriver:144]: Login response has been received```

[djre4orm]

Hello all, posting my result today with the same 429 error. It started yesterday when fare checking. However in the logs I believe I had a successful fare check two days ago. This is from a VM running locally on my network, verizon IP.

2024-01-31 13:26:09 DEBUG Process-1[fare_checker:50]: Found 4 matching flights
2024-01-31 13:26:09 DEBUG Process-1[fare_checker:37]: Flight price change found for +15877 PTS
2024-01-31 13:26:09 DEBUG Process-1[fare_checker:30]: Checking current price for flight
2024-01-31 13:26:09 DEBUG Process-1[fare_checker:88]: Fetching reservation information
2024-01-31 13:26:11 DEBUG Process-1[utils:34]: Successfully made request after 1 attempts
2024-01-31 13:26:11 DEBUG Process-1[fare_checker:103]: Retrieving search information for the current flight
2024-01-31 13:26:18 DEBUG Process-1[utils:34]: Successfully made request after 1 attempts
2024-01-31 13:26:18 DEBUG Process-1[fare_checker:82]: Retrieving matching flights
2024-01-31 13:26:19 DEBUG Process-1[utils:34]: Successfully made request after 1 attempts
2024-01-31 13:26:19 DEBUG Process-1[fare_checker:50]: Found 6 matching flights
2024-01-31 13:26:19 DEBUG Process-1[fare_checker:37]: Flight price change found for 0 PTS
2024-01-31 13:26:19 DEBUG Process-1[reservation_monitor:80]: Lock released
2024-01-31 13:26:19 DEBUG Process-1[reservation_monitor:116]: Sleeping for 86370 seconds

2024-02-01 13:26:09 DEBUG Process-1[fare_checker:30]: Checking current price for flight
2024-02-01 13:26:09 DEBUG Process-1[fare_checker:88]: Fetching reservation information
2024-02-01 13:26:11 DEBUG Process-1[utils:34]: Successfully made request after 1 attempts
2024-02-01 13:26:11 DEBUG Process-1[fare_checker:103]: Retrieving search information for the current flight
2024-02-01 13:26:13 DEBUG Process-1[utils:34]: Successfully made request after 1 attempts
2024-02-01 13:26:13 DEBUG Process-1[fare_checker:82]: Retrieving matching flights
2024-02-01 13:26:18 DEBUG Process-1[utils:41]: Failed to make request after 7 attempts: Too Many Requests 429
2024-02-01 13:26:18 DEBUG Process-1[utils:44]: Response body: {
    "code": 429999999,

    "message": "Error.",

    "messageKey": "ERROR",

    "httpStatusCode": "BAD_REQUEST",

    "requestId": "",

    "infoList": []
   }
2024-02-01 13:26:18 ERROR Process-1[reservation_monitor:102]: Requesting error during fare check. Too Many Requests 429. Skipping...
2024-02-01 13:26:18 DEBUG Process-1[reservation_monitor:80]: Lock released
2024-02-01 13:26:18 DEBUG Process-1[reservation_monitor:116]: Sleeping for 86371 seconds

2024-02-01 16:48:31 ERROR Process-1[reservation_monitor:102]: Requesting error during fare check. Too Many Requests 429. Skipping...
2024-02-01 16:48:31 DEBUG Process-1[reservation_monitor:80]: Lock released
2024-02-01 16:48:31 DEBUG Process-1[reservation_monitor:116]: Sleeping for 86373 seconds
2024-02-02 12:09:55 DEBUG Process-1:1[checkin_handler:94]: Acquiring lock...
2024-02-02 12:09:55 DEBUG Process-1:1[checkin_handler:96]: Lock acquired
2024-02-02 12:09:55 DEBUG Process-1:1[checkin_scheduler:50]: Refreshing headers for current session
2024-02-02 12:09:55 DEBUG Process-1:1[webdriver:103]: Starting webdriver for current session
2024-02-02 12:09:57 DEBUG Process-1:1[webdriver:119]: Using browser version: 109.0.5414.119
2024-02-02 12:09:57 DEBUG Process-1:1[webdriver:123]: Loading Southwest Check-In page
2024-02-02 12:10:02 DEBUG Process-1:1[webdriver:65]: Waiting for valid headers
2024-02-02 12:10:02 DEBUG Process-1:1[webdriver:152]: Waiting for headers_set to be set
2024-02-02 12:10:02 DEBUG Process-1:1[webdriver:156]: headers_set set successfully
2024-02-02 12:10:03 DEBUG Process-1:1[checkin_handler:99]: Lock released
2024-02-02 12:10:03 DEBUG Process-1:1[checkin_handler:103]: Sleeping until check-in: 1791 seconds...
2024-02-02 12:39:55 DEBUG Process-1:1[checkin_handler:124]: Attempting to check in
2024-02-02 12:39:55 DEBUG Process-1:1[checkin_handler:138]: Making GET request to check in
2024-02-02 12:40:08 DEBUG Process-1:1[utils:41]: Failed to make request after 20 attempts: Too Many Requests 429
2024-02-02 12:40:08 DEBUG Process-1:1[utils:44]: Response body: {
    "code": 429999999,

    "message": "Error.",

    "messageKey": "ERROR",

    "httpStatusCode": "BAD_REQUEST",

    "requestId": "",

    "infoList": []
   }
2024-02-02 12:40:08 DEBUG Process-1:1[checkin_handler:147]: Failed to check in. Error: Too Many Requests 429. Exiting
2024-02-02 12:40:08 DEBUG Process-1:1[notification_handler:108]: Sending failed check-in notification...

[jdholtz]

@djre4orm are you using Google Chrome or Chromium?

Edit: Never mind, the error you are receiving won’t be affected by the browser (that’s only for logging in right now)

[djre4orm]

@djre4orm are you using Google Chrome or Chromium?

Google Chrome 109.0.5414.119

[dmytrokoren]

@jdholtz can we try to resize the browser window in your -develop branch? I can pull that and try in docker I have one reservation coming up, so we can get away from mobile version site and use desktop. I think this can resolve the issue. IMHO

[camjones1708]

Edit:This is for check-in only FYI

DEBUG:lib.checkin_handler:Lock released
2024-02-02 11:40:35 DEBUG Process-1:1[checkin_handler:103]: Sleeping until check-in: 1759 seconds...
DEBUG:lib.checkin_handler:Sleeping until check-in: 1759 seconds...
2024-02-02 12:09:55 DEBUG Process-1:1[checkin_handler:124]: Attempting to check in
DEBUG:lib.checkin_handler:Attempting to check in
Checking in to flight from 'SRC' to 'DEST' for <REDACTED>

2024-02-02 12:09:55 DEBUG Process-1:1[checkin_handler:138]: Making GET request to check in
DEBUG:lib.checkin_handler:Making GET request to check in
2024-02-02 12:10:12 DEBUG Process-1:1[utils:41]: Failed to make request after 20 attempts: Too Many Requests 429
DEBUG:lib.utils:Failed to make request after 20 attempts: Too Many Requests 429
2024-02-02 12:10:12 DEBUG Process-1:1[utils:44]: Response body: {
    "code": 429999999,

    "message": "Error.",

    "messageKey": "ERROR",

    "httpStatusCode": "BAD_REQUEST",

    "requestId": "",

    "infoList": []
   }
DEBUG:lib.utils:Response body: {
    "code": 429999999,

    "message": "Error.",

    "messageKey": "ERROR",

    "httpStatusCode": "BAD_REQUEST",

    "requestId": "",

    "infoList": []
   }
2024-02-02 12:10:12 DEBUG Process-1:1[checkin_handler:147]: Failed to check in. Error: Too Many Requests 429. Exiting
DEBUG:lib.checkin_handler:Failed to check in. Error: Too Many Requests 429. Exiting
2024-02-02 12:10:12 DEBUG Process-1:1[notification_handler:108]: Sending failed check-in notification...
DEBUG:lib.notification_handler:Sending failed check-in notification...
Failed to check in to flight <CONF> for <REDACTED>. Reason: Too Many Requests 429.
Check in at this url: https://mobile.southwest.com/check-in

I was attempting to run from console rather than from docker on raspi 4 (arm64) and was getting the same error. I also ran on my os x and had zero issues.

[jdholtz]

@jdholtz can we try to resize the browser window in your -develop branch? I can pull that and try in docker I have one reservation coming up, so we can get away from mobile version site and use desktop. I think this can resolve the issue. IMHO

@dmytrokoren yes, I believe you can change the window size by passing in `chromium_arg="--window-size=1920x1080" into the Driver constructor (or whatever size you want).

I haven't tried the desktop version of the website, but I will try that to see if it resolves the issue.

[sevenlayercookie]

I'm not sure if this is related or coincidental, but my server crashed due to storage space being filled, and I see that my /var/tmp folder is filled of 86 gb of chromium temp files related to the southwest script. Perhaps there is something misconfigured with my install (Docker), but it implies to me an excessive number of requests being made to Southwest.

[jdholtz]

Perhaps there is something misconfigured with my install (Docker), but it implies to me an excessive number of requests being made to Southwest.

That's really strange. Could we talk about this in a Discussion?

[sevenlayercookie]

yes I'll open a new discussion

[nathan921]

I don't know if it's useful but I just upgrade from Unraid 6.12.3 to 6.12.6 and the scheduling works again in docker. Is there any way I can force a fair check or checkin and see if that's working as well?

[jdholtz]

Is there any way I can force a fair check or checkin and see if that's working as well?

By enabling check_fares in your config (or by removing it as it defaults to true in v7.1), that will run a fare check on your flights. You can run the script with the --verbose flag to get more output. Currently, the script fails after the line Retrieving matching flights so if it successfully retrieves the flights, then it is working again.

[wpkato]

I just ran develop in a console and was able to login, check flight and fares also. Very weird

Within docker, develop still getting a 429 error

On Tue, Feb 6, 2024 at 11:18 AM Joey Holtzman @.***> wrote:

Is there any way I can force a fair check or checkin and see if that's working as well?

By enabling check_fares in your config (or by removing it as it defaults to true in v7.1), that will run a fare check on your flights. You can run the script with the --verbose flag to get more output. Currently, the script fails after the line Retrieving matching flights so if it successfully retrieves the flights, then it is working again.

— Reply to this email directly, view it on GitHub https://github.com/jdholtz/auto-southwest-check-in/issues/201#issuecomment-1930404820, or unsubscribe https://github.com/notifications/unsubscribe-auth/A2QJTUY5BTIAIW2IFJCTL6LYSJQXHAVCNFSM6AAAAABBTESKY6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZQGQYDIOBSGA . You are receiving this because you commented.Message ID: @.***>

[boonep]

I pulled the latest develop and spun it up in a docker container and was able to successfully login/compare fares/schedule check-in.

Prior to that, I was consistently getting the 429 error on v6.1

[jchavesc]

I've gotten this issue twice this week, but only when I ran it on AWS instances (Ubuntu 22.04). On a home server also on Ubuntu 22.04, had no issues also last week.

[eiiot]

Still running into this issue with develop and docker compose.

root@ubuntu-4gb-ash-1:~/docker# docker logs auto-southwest
Encountered a Too Many Requests error while logging in. Skipping reservation retrieval
Encountered a Too Many Requests error while logging in. Skipping reservation retrieval
Encountered a Too Many Requests error while logging in. Skipping reservation retrieval
Encountered a Too Many Requests error while logging in. Skipping reservation retrieval

Edit: and also when running the develop branch with python.

2024-02-07 18:21:58 DEBUG MainProcess[log:23]: Initialized the application
2024-02-07 18:21:58 DEBUG MainProcess[main:112]: Auto-Southwest Check-In v7.1
2024-02-07 18:21:58 DEBUG MainProcess[main:113]: Called with 0 arguments
2024-02-07 18:21:58 DEBUG MainProcess[config:108]: Initializing configuration file
2024-02-07 18:21:58 DEBUG MainProcess[config:137]: Reading the configuration file
2024-02-07 18:21:58 DEBUG MainProcess[config:150]: Reading configuration from environment variables
2024-02-07 18:21:58 DEBUG MainProcess[config:54]: Setting check fares to False
2024-02-07 18:21:58 DEBUG MainProcess[config:68]: Setting notification level to <NotificationLevel.INFO: 1>
2024-02-07 18:21:58 DEBUG MainProcess[config:81]: Using 1 notification services
2024-02-07 18:21:58 DEBUG MainProcess[config:85]: Setting retrieval interval to 1 hours
2024-02-07 18:21:58 DEBUG MainProcess[config:120]: Creating configurations for 2 accounts
2024-02-07 18:21:58 DEBUG MainProcess[config:127]: Creating configurations for 0 reservations
2024-02-07 18:21:58 DEBUG MainProcess[main:142]: Monitoring 2 accounts and 0 reservations
2024-02-07 18:21:58 DEBUG Process-1[reservation_monitor:154]: Acquiring lock...
2024-02-07 18:21:58 DEBUG Process-1[reservation_monitor:156]: Lock acquired
2024-02-07 18:21:58 DEBUG Process-1[reservation_monitor:178]: Retrieving reservations for account
2024-02-07 18:21:58 DEBUG Process-1[webdriver:103]: Starting webdriver for current session
2024-02-07 18:21:58 DEBUG Process-2[reservation_monitor:154]: Acquiring lock...
2024-02-07 18:23:00 DEBUG Process-1[webdriver:119]: Using browser version: 121.0.6167.139
2024-02-07 18:23:00 DEBUG Process-1[webdriver:123]: Loading Southwest Check-In page
2024-02-07 18:23:04 DEBUG Process-1[webdriver:80]: Logging into account to get a list of reservations and valid headers
2024-02-07 18:23:07 DEBUG Process-1[webdriver:152]: Waiting for headers_set to be set
2024-02-07 18:23:07 DEBUG Process-1[webdriver:156]: headers_set set successfully
2024-02-07 18:23:07 DEBUG Process-1[webdriver:152]: Waiting for login_request_id to be set
2024-02-07 18:23:08 DEBUG Process-1[webdriver:144]: Login response has been received
2024-02-07 18:23:08 DEBUG Process-1[webdriver:156]: login_request_id set successfully
2024-02-07 18:23:08 DEBUG Process-1[webdriver:212]: Logging in failed for an unknown reason
2024-02-07 18:23:08 WARNING Process-1[reservation_monitor:187]: Encountered a Too Many Requests error while logging in. Skipping reservation retrieval
2024-02-07 18:23:08 DEBUG Process-1[reservation_monitor:167]: Lock released
2024-02-07 18:23:08 DEBUG Process-2[reservation_monitor:156]: Lock acquired
2024-02-07 18:23:08 DEBUG Process-2[reservation_monitor:178]: Retrieving reservations for account
2024-02-07 18:23:08 DEBUG Process-1[reservation_monitor:116]: Sleeping for 3529 seconds
2024-02-07 18:23:08 DEBUG Process-2[webdriver:103]: Starting webdriver for current session

I've gotten this issue twice this week, but only when I ran it on AWS instances (Ubuntu 22.04). On a home server also on Ubuntu 22.04, had no issues also last week.

Just ran the develop container on my home server as well, ran into the same issue:

eliot@Eliot-Server:~/docker-2$ docker-compose logs auto-southwest
Attaching to auto-southwest
auto-southwest    | Encountered a Too Many Requests error while logging in. Skipping reservation retrieval

[jdholtz]

This is still an issue on v7.2. I've spent a lot of time trying to find out what the issue is and fix it. The issue is that the Southwest website is giving invalid request headers to the browser when it loads the webpage. However, I have not found any indication as to how Southwest is detecting the browser as a bot and have not been able to reproduce it outside of Docker (although others have).

Edit: The browser passes a bot detection test on https://nowsecure.nl/ in the Docker container, which makes the invalid request headers issue even more strange (Southwest's website is not very good at detecting bots, at least up until now potentially).

[eiiot]

I have not found any indication as to how Southwest is detecting the browser as a bot and have not been able to reproduce it outside of Docker (although others have).

@jdholtz can I help with any logging? I have this issue right now with develop and the python script.

[jdholtz]

@jdholtz can I help with any logging? I have this issue right now with develop and the python script.

Unfortunately the logs won't tell what the issue is beyond a 429, so they aren't very helpful for fixing this issue.

[Royal2000H]

I would think the primary difference between why it may work as a python script vs in Docker is something with the browser environment. So potentially Southwest is fingerprinting different browser characteristics, user agents, extensions, window size, javascript, etc. There are different headless browser detectors that this can be tested against. And different browser options too (playwright vs selenium, etc).

[eiiot]

Annoyingly, I also can't recreate the issue on my local machine, it's only a problem on servers or with docker... could it be something with the Southwest website detecting the User-Agent of headless chrome / chromium?

[joewesch]

I tried messing around with the User Agent and headers the other night to no avail. Just as a data point, I also tried adding is_mobile=True to the Driver instantiation inside the docker container as well as on my Mac (which was working just fine) to change the User Agent to emulate a Pixel Phone and it caused a 429 on both. So, I was able to replicate the inverse at least - cause a 429 outside of docker.

[hildebrau]

The issue is that the Southwest website is giving invalid request headers to the browser when it loads the webpage.

How sure are you that it's Southwest giving invalid request headers, versus docker environment having some permissions issue and it can't write the request headers properly to storage? Has anyone tried forcing the header request routines to be wrapped in sudo for grins?

[conor888]

I'm not sure if this is related or coincidental, but my server crashed due to storage space being filled, and I see that my /var/tmp folder is filled of 86 gb of chromium temp files related to the southwest script. Perhaps there is something misconfigured with my install (Docker), but it implies to me an excessive number of requests being made to Southwest.

I just had this happen today, my entire server disk filled with logs. Just ended up purging the docker container. Interestingly when I started the container this morning (I tried restarting the container the day of a flight check-in as that brought me success last time a couple of weeks ago), I never got a notification that sign-in completed, but I didn't notice. I also never got an error. Perhaps it was trying to sign-in over and over again infinitely?

[wpkato]

• This is while running script in console under screen command
• I am still unsuccessful in getting a running version in docker.

While in running in a console, it ran with no errors even executing multiple test restarts. Today however it failed to automatically do the fare check at the 24hr mark. Failed with a 429.

I then recreated the environment in a different instance using VirtualBox in a Linux Mint 21.3 instance (based on Ubuntu 22,04) and was able to get the script to run even after multiple test restarts. One of the major differences was that this LM instance was running 121.x version of Chromium and my home server was running Debian 11 w/ the 120.x version. I didn't feel like upgrading my Debian instance to 12 just for this test so I uninstalled Chromium and installed google-chrome-stable instead which is on the 121.x version. This then worked, again after multiple test restarts with no errors

Not sure if the above is really relevant to the issue but thought I would document it.

[jdholtz]

So potentially Southwest is fingerprinting different browser characteristics, user agents, extensions, window size, javascript, etc

@Royal2000H Yeah, that's probably most likely the issue. Now we just need to find what exactly it is fingerprinting to determine the bot...

And different browser options too (playwright vs selenium, etc).

I did try Playwright at one point, but there aren't any libraries that match SeleniumBase for going undetected with Playwright.

could it be something with the Southwest website detecting the User-Agent of headless chrome / chromium?

@eiiot SeleniumBase does clean the user agent correctly. I also looked at it myself and it is using the exact same agent as my browser (different version numbers, but same exact characteristics).

I tried messing around with the User Agent and headers the other night to no avail. Just as a data point, I also tried adding is_mobile=True to the Driver instantiation...So, I was able to replicate the inverse at least - cause a 429 outside of docker.

@joewesch that's a really good idea. Unfortunately I did not get a 429 when I added that to the driver instantiation, but that does give a better idea as to what characteristics the Southwest website is looking for to detect bots.

How sure are you that it's Southwest giving invalid request headers, versus docker environment having some permissions issue and it can't write the request headers properly to storage? Has anyone tried forcing the header request routines to be wrapped in sudo for grins?

@hildebrau I saved the headers from the Docker container and my host. When I had the script use the headers from my host machine, it worked. When I had my host use the Docker headers, it did not work. No header information seems to be missing (all header keys are the same and the values are very similar in length). The script also does run as root user in the Docker container (will try changing that at a later point as it isn't good practice), so there shouldn't be any issues with the browser running into permission issues.

[jdholtz]

Perhaps it was trying to sign-in over and over again infinitely?

@conor888 it will only ever try to sign in once or fail and wait until the next retrieval interval. Since it is hanging, that is most likely the browser failing to load for some reason.

[edmcman]

Hi, I just wanted to say that I'm encountering this problem outside of docker without changing anything. I'm using master, not develop.

I only seem to get a 429 on fare checks though. Logins are ok.

[ba221400]

I just experienced this as well. Adding some comments in the event it is helpful.

Running locally (no docker) on my Macbook worked correctly, no 429 errors--fare check and check-in were successful.

I set up a container using docker compose on an EC2 instance (us-east-2) which threw the 429 error when running the fare check. I set the fare check to false which avoided the error there, but the check-in subsequently failed. I assumed this might be due to it being in AWS, so I moved the docker compose and config.json files to a local linux server. I've already checked in, so I don't know if that will work or not, but the fare check is also failing there with the 429 error.

[edmcman]

My checkin also failed with 429.

2024-02-10 16:45:28 DEBUG Process-1:1[checkin_handler:103]: Sleeping until check-in: 1766 seconds...
2024-02-10 17:14:55 DEBUG Process-1:1[checkin_handler:124]: Attempting to check in                         
Checking in to flight from 'Denver' to 'Philadelphia' for Edward Schwartz                    

2024-02-10 17:14:55 DEBUG Process-1:1[checkin_handler:138]: Making GET request to check in                             2024-02-10 17:15:09 DEBUG Process-1:1[utils:39]: Failed to make request after 20 attempts: Too Many Requests 429       2024-02-10 17:15:09 DEBUG Process-1:1[utils:42]: Response body: {
    "code": 429999999,                                                                                                                              
    "message": "Error.",                                                                                               

    "messageKey": "ERROR",                                                                                             

    "httpStatusCode": "BAD_REQUEST",                                                                                   

    "requestId": "",                                                                                                   

    "infoList": []                                                                                                     
   }                                                                                                                   
2024-02-10 17:15:09 DEBUG Process-1:1[checkin_handler:147]: Failed to check in. Error: Too Many Requests 429. Exiting
2024-02-10 17:15:09 DEBUG Process-1:1[notification_handler:109]: Sending failed check-in notification...
Failed to check in to flight 4ZZ4I3 for Edward Schwartz. Reason: Too Many Requests 429.  
Check in at this url: https://mobile.southwest.com/check-in                    

[Royal2000H]

Adding more info, tried testing via changedetection in Docker. Tried the mobile login endpoints using Playright chromium as well as the plain browser and Southwest rejected it. Also tried using browser steps to go login through the full site. Also rejected. Of course, logging in with a real browser manually from the same IP is logging in just fine.

[wpkato]

RESOLVED: Took the special characters out in the password that previously worked just fine for weeks and now the script is working again in console

Now I received a new error that I haven't seen yet...

2024-02-12 21:56:04 DEBUG MainProcess[log:23]: Initialized the application 2024-02-12 21:56:04 DEBUG MainProcess[main:112]: Auto-Southwest Check-In v7.2 2024-02-12 21:56:04 DEBUG MainProcess[main:113]: Called with 2 arguments 2024-02-12 21:56:04 DEBUG MainProcess[config:108]: Initializing configuration file 2024-02-12 21:56:04 DEBUG MainProcess[config:137]: Reading the configuration file 2024-02-12 21:56:04 DEBUG MainProcess[config:141]: No configuration file found. Using defaults 2024-02-12 21:56:04 DEBUG MainProcess[config:151]: Reading configuration from environment variables 2024-02-12 21:56:04 DEBUG MainProcess[config:120]: Creating configurations for 1 accounts 2024-02-12 21:56:04 DEBUG MainProcess[main:129]: Account added through CLI arguments 2024-02-12 21:56:04 DEBUG MainProcess[main:142]: Monitoring 1 accounts and 0 reservations 2024-02-12 21:56:05 DEBUG Process-1[reservation_monitor:154]: Acquiring lock... 2024-02-12 21:56:05 DEBUG Process-1[reservation_monitor:156]: Lock acquired 2024-02-12 21:56:05 DEBUG Process-1[reservation_monitor:178]: Retrieving reservations for account 2024-02-12 21:56:05 DEBUG Process-1[webdriver:103]: Starting webdriver for current session 2024-02-12 21:56:27 DEBUG Process-1[webdriver:119]: Using browser version: 121.0.6167.160 2024-02-12 21:56:27 DEBUG Process-1[webdriver:123]: Loading Southwest Check-In page 2024-02-12 21:56:32 DEBUG Process-1[webdriver:80]: Logging into account to get a list of reservations and valid headers 2024-02-12 21:56:33 DEBUG Process-1[webdriver:152]: Waiting for headers_set to be set 2024-02-12 21:56:33 DEBUG Process-1[webdriver:156]: headers_set set successfully 2024-02-12 21:56:35 DEBUG Process-1[webdriver:152]: Waiting for login_request_id to be set 2024-02-12 21:56:35 DEBUG Process-1[webdriver:144]: Login response has been received 2024-02-12 21:56:35 DEBUG Process-1[webdriver:156]: login_request_id set successfully 2024-02-12 21:56:35 DEBUG Process-1[webdriver:209]: Invalid credentials provided when attempting to log in 2024-02-12 21:56:35 DEBUG Process-1[reservation_monitor:193]: Error logging in. Reason: Invalid credentials. Status code: 400. Exiting 2024-02-12 21:56:35 DEBUG Process-1[notification_handler:83]: Sending failed login notification... Failed to log in to account with username Xxxxxxx. Reason: Invalid credentials. Status code: 400.

My credentials haven't changed and I'm able to log in with no issues via a web browser in the desktop OS.

[nathan921]

RESOLVED: Took the special characters out that were previously work just fine for weeks and now the script is working again in console

Now I received a new error that I haven't seen yet...

2024-02-12 21:56:04 DEBUG MainProcess[log:23]: Initialized the application 2024-02-12 21:56:04 DEBUG MainProcess[main:112]: Auto-Southwest Check-In v7.2 2024-02-12 21:56:04 DEBUG MainProcess[main:113]: Called with 2 arguments 2024-02-12 21:56:04 DEBUG MainProcess[config:108]: Initializing configuration file 2024-02-12 21:56:04 DEBUG MainProcess[config:137]: Reading the configuration file 2024-02-12 21:56:04 DEBUG MainProcess[config:141]: No configuration file found. Using defaults 2024-02-12 21:56:04 DEBUG MainProcess[config:151]: Reading configuration from environment variables 2024-02-12 21:56:04 DEBUG MainProcess[config:120]: Creating configurations for 1 accounts 2024-02-12 21:56:04 DEBUG MainProcess[main:129]: Account added through CLI arguments 2024-02-12 21:56:04 DEBUG MainProcess[main:142]: Monitoring 1 accounts and 0 reservations 2024-02-12 21:56:05 DEBUG Process-1[reservation_monitor:154]: Acquiring lock... 2024-02-12 21:56:05 DEBUG Process-1[reservation_monitor:156]: Lock acquired 2024-02-12 21:56:05 DEBUG Process-1[reservation_monitor:178]: Retrieving reservations for account 2024-02-12 21:56:05 DEBUG Process-1[webdriver:103]: Starting webdriver for current session 2024-02-12 21:56:27 DEBUG Process-1[webdriver:119]: Using browser version: 121.0.6167.160 2024-02-12 21:56:27 DEBUG Process-1[webdriver:123]: Loading Southwest Check-In page 2024-02-12 21:56:32 DEBUG Process-1[webdriver:80]: Logging into account to get a list of reservations and valid headers 2024-02-12 21:56:33 DEBUG Process-1[webdriver:152]: Waiting for headers_set to be set 2024-02-12 21:56:33 DEBUG Process-1[webdriver:156]: headers_set set successfully 2024-02-12 21:56:35 DEBUG Process-1[webdriver:152]: Waiting for login_request_id to be set 2024-02-12 21:56:35 DEBUG Process-1[webdriver:144]: Login response has been received 2024-02-12 21:56:35 DEBUG Process-1[webdriver:156]: login_request_id set successfully 2024-02-12 21:56:35 DEBUG Process-1[webdriver:209]: Invalid credentials provided when attempting to log in 2024-02-12 21:56:35 DEBUG Process-1[reservation_monitor:193]: Error logging in. Reason: Invalid credentials. Status code: 400. Exiting 2024-02-12 21:56:35 DEBUG Process-1[notification_handler:83]: Sending failed login notification... Failed to log in to account with username Xxxxxxx. Reason: Invalid credentials. Status code: 400.

My credentials haven't changed and I'm able to log in with no issues via a web browser in the desktop OS.

Thanks for finding this. I resolved my issue as well by removing a special character. What's interesting is that I had swapped from using my account, which had the special character, to my name and reservation number, which didn't, and it was still giving me a 400 error before this fix.

[borski]

Which special characters? Just so I don't have to binary search my way through my long password lol

[nathan921]

mine was an asterisk *, but i'm sure it effects all the specials characters they accept in passwords.

[lukeallister]

I changed my password to alphanumeric and am still getting the same 'encountered too many requests error' with docker.

[wpkato]

This is a solution to a different error (error 400) not the 429 error. Script with Docker and error 429 is still still unresolved

On Wed, Feb 14, 2024, 15:51 Luke Miller @.***> wrote:

I changed my password to alphanumeric and am still getting the same 'encountered too many requests error' with docker.

— Reply to this email directly, view it on GitHub https://github.com/jdholtz/auto-southwest-check-in/issues/201#issuecomment-1944722778, or unsubscribe https://github.com/notifications/unsubscribe-auth/A2QJTU4MAGZFL7RMLKTOJBDYTUWXJAVCNFSM6AAAAABBTESKY6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBUG4ZDENZXHA . You are receiving this because you commented.Message ID: @.***>

[Royal2000H]

I'm unable to try this myself at this time, but if anyone wants to play around, a next step to explore is to try to login to southwest using: https://github.com/ultrafunkamsterdam/undetected-chromedriver

If that works, then integration with this project would be promising.

[edmcman]

Isn't it already doing so?

https://github.com/jdholtz/auto-southwest-check-in/blob/6464ad6261a2d302b1a0765b49d75b525bb66e05/lib/webdriver.py#L117