jdholtz / auto-southwest-check-in

A Python script that automatically checks in to your Southwest flight 24 hours beforehand.
GNU General Public License v3.0
467 stars 89 forks source link

Getting error 429 during fare checks and logins #201

Closed StevenMassaro closed 5 months ago

StevenMassaro commented 11 months ago

Version

Auto-Southwest Check-In v7.1

Browser Version

Docker image

Description

Recently, I've started getting these 429 errors when checking the prices for upcoming flights. I am not sure if they are truly a 429 error, caused by actually sending too many requests, or a different error being reported as a 429.

To Reproduce

  1. Add flights to the app
  2. Check for price differences

Expected Behavior

No 429 error

Relevant logs and program output

2024-01-09 13:54:00 DEBUG Process-3[fare_checker:30]: Checking current price for flight
2024-01-09 13:54:00 DEBUG Process-3[fare_checker:88]: Fetching reservation information
2024-01-09 13:54:02 DEBUG Process-3[utils:34]: Successfully made request after 1 attempts
2024-01-09 13:54:02 DEBUG Process-3[fare_checker:103]: Retrieving search information for the current flight
2024-01-09 13:54:03 DEBUG Process-3[utils:34]: Successfully made request after 1 attempts
2024-01-09 13:54:03 DEBUG Process-3[fare_checker:82]: Retrieving matching flights
2024-01-09 13:54:08 DEBUG Process-3[utils:41]: Failed to make request after 7 attempts: Too Many Requests 429
2024-01-09 13:54:08 DEBUG Process-3[utils:44]: Response body: {
    "code": 429999999,

    "message": "Error.",

    "messageKey": "ERROR",

    "httpStatusCode": "BAD_REQUEST",

    "requestId": "",

    "infoList": []
   }
2024-01-09 13:54:08 ERROR Process-3[reservation_monitor:102]: Requesting error during fare check. Too Many Requests 429. Skipping...

Additional context

No response

jdholtz commented 11 months ago

Thanks for reporting this. Unfortunately, I am unable to reproduce this issue. Can you reproduce this with the develop image too?

A 429 Error usually happens when Southwest is detecting that you are using a bot. Are you using a VPN on the computer running the script? Many people have gotten 429 errors before, so there may be some common troubleshooting steps in past issues for you to try.

What I noticed is that the request it fails at is a POST request. The only other time a POST request is sent is during check-in, so it may fail to check in for you too.

StevenMassaro commented 11 months ago

I am able to reproduce this with the develop branch.

I am not using a VPN.

I think I'll wait until tomorrow and see if I continue to get this error tomorrow. Maybe it's a fluke.

StevenMassaro commented 11 months ago

I don't know if it's related, but the first flight (of the four that it attempts to check) has an error like this:

2024-01-09 19:04:58 DEBUG Process-3[reservation_monitor:104]: Flight cannot be changed online. Skipping fare check

caused by the flight being rescheduled by Southwest.

Maybe this has downstream effect to the later requests that get sent? I am not sure on that. I have seen it successfully make at least one or two successful fare checks since I started seeing the errors.

StevenMassaro commented 11 months ago

FWIW I get the same error on v6.1.

jdholtz commented 10 months ago

I don't know if it's related, but the first flight (of the four that it attempts to check) has an error like this:

That just means the flight's fares cannot be checked because the Southwest website doesn't allow you to go in and view the same flight's fares but it is not the cause of the issue. There was an issue with flight rescheduling in this script, but that has been fixed on the latest develop commit.

This may be a random issue that happens occasionally (I'm unsure what and why it occurs though). We can wait for a few days and see if it is indeed a fluke.

StevenMassaro commented 10 months ago

This may be a random issue that happens occasionally (I'm unsure what and why it occurs though). We can wait for a few days and see if it is indeed a fluke.

I think that's what we should do. Running the script on a different computer has no issues. Granted the other computer is Windows (instead of Linux) and the script is running natively (not in docker). Nevertheless, it's not like my IP is banned because both computers are on the same network.

I'll report back later.

StevenMassaro commented 10 months ago

Some new theories.

I restarted the docker container this morning. The first three flights that it checked were successful. The following five flights that it attempted to check all got 429 errors. Of those five, the last four were in a second account.

I restarted it again. This time, the fare check was successful for all of the flights in the first account, but failed for the flights in the second account (4 success and 4 fails).

I did print out the response headers on the failing requests and unfortunately there is no key telling you when the rate limit resets.

{'Content-Type': 'text/plain; charset=UTF-8', 'Content-Length': '163', 'X-Ion-Hop': '1', 'Expires': 'Wed, 10 Jan 2024 13:19:15 GMT', 'Cache-Control': 'max-age=0, no-cache, no-store', 'Pragma': 'no-cache', 'Date': 'Wed, 10 Jan 2024 13:19:15 GMT', 'Connection': 'close', 'Set-Cookie': '<removed>; Domain=southwest.com; Secure', 'X-Content-Type-Options': 'nosniff', 'Content-Security-Policy': "frame-ancestors 'self';", 'Strict-Transport-Security': 'max-age=31536000', 'terms-of-service': "Unauthorized access, display, or use of Southwest's Company Information, including fare data, is prohibited by the Terms & Conditions on Southwest.com and Swabiz.com."}

Note that these issues all occurred on my linux home server.

I decided to run it again on my Windows laptop. I reliably get the same errors when running the docker container on my laptop too. However, running the python program outside of the docker container works, every time.

Thinking that maybe it was my home internet somehow, I tried again with my laptop and docker container on my cell phone hotspot - same 429 error.

It really seems to me like there is something about the docker container that is being detected by Southwest. I am open to debugging this further but I am wondering if you have any ideas of what to look at next. I am also a bit surprised you're not seeing this.

StevenMassaro commented 10 months ago

One more thing I just tried. I tried adding in 90 second sleeps between the retries in util.py and a 90 second sleep in fare_checker.py _get_matching_flights right before it does the search. I still get a 429 error when running in docker.

sevenlayercookie commented 10 months ago

I have started getting 429 errors in the last week or so as well. I am running the script in a docker container on my Linux home server. I get the error for fare checks and this morning I got it for flight check-in unfortunately. It logs into my accounts (two accounts) with no issues. Is there something about the Docker container that Southwest is picking up on?

Edit: running v7.1, no VPN or proxy

Log

```YAML 2024-01-10 17:51:38 DEBUG MainProcess[log:23]: Initialized the application 2024-01-10 17:51:38 DEBUG MainProcess[main:112]: Auto-Southwest Check-In v7.1 2024-01-10 17:51:38 DEBUG MainProcess[main:113]: Called with 0 arguments 2024-01-10 17:51:38 DEBUG MainProcess[config:107]: Initializing configuration file 2024-01-10 17:51:38 DEBUG MainProcess[config:135]: Reading the configuration file 2024-01-10 17:51:38 DEBUG MainProcess[config:53]: Setting check fares to True 2024-01-10 17:51:38 DEBUG MainProcess[config:67]: Setting notification level to 2024-01-10 17:51:38 DEBUG MainProcess[config:80]: Using 2 notification services 2024-01-10 17:51:38 DEBUG MainProcess[config:84]: Setting retrieval interval to 24 hours 2024-01-10 17:51:38 DEBUG MainProcess[config:118]: Creating configurations for 2 accounts 2024-01-10 17:51:38 DEBUG MainProcess[config:125]: Creating configurations for 0 reservations 2024-01-10 17:51:38 DEBUG MainProcess[main:142]: Monitoring 2 accounts and 0 reservations 2024-01-10 17:51:39 DEBUG Process-1[reservation_monitor:154]: Acquiring lock... 2024-01-10 17:51:39 DEBUG Process-1[reservation_monitor:156]: Lock acquired 2024-01-10 17:51:39 DEBUG Process-1[reservation_monitor:178]: Retrieving reservations for account 2024-01-10 17:51:39 DEBUG Process-1[webdriver:103]: Starting webdriver for current session 2024-01-10 17:51:39 DEBUG Process-2[reservation_monitor:154]: Acquiring lock... 2024-01-10 17:51:42 DEBUG Process-1[webdriver:119]: Using browser version: 117.0.5938.62 2024-01-10 17:51:42 DEBUG Process-1[webdriver:123]: Loading Southwest Check-In page 2024-01-10 17:51:55 DEBUG Process-1[webdriver:80]: Logging into account to get a list of reservations and valid headers 2024-01-10 17:51:59 DEBUG Process-1[webdriver:152]: Waiting for headers_set to be set 2024-01-10 17:51:59 DEBUG Process-1[webdriver:156]: headers_set set successfully 2024-01-10 17:52:03 DEBUG Process-1[webdriver:144]: Login response has been received 2024-01-10 17:52:03 DEBUG Process-1[webdriver:152]: Waiting for login_request_id to be set 2024-01-10 17:52:03 DEBUG Process-1[webdriver:156]: login_request_id set successfully 2024-01-10 17:52:03 DEBUG Process-1[webdriver:226]: First time logging in. Setting account name Successfully logged in to John Doe's account 2024-01-10 17:52:03 DEBUG Process-1[webdriver:152]: Waiting for trips_request_id to be set 2024-01-10 17:52:04 DEBUG Process-1[webdriver:148]: Upcoming trips response has been received 2024-01-10 17:52:04 DEBUG Process-1[webdriver:156]: trips_request_id set successfully 2024-01-10 17:52:05 DEBUG Process-1[reservation_monitor:197]: Successfully retrieved 1 reservations 2024-01-10 17:52:05 DEBUG Process-1[reservation_monitor:84]: Scheduling flight check-ins for 1 reservations 2024-01-10 17:52:05 DEBUG Process-1[checkin_scheduler:76]: Retrieving reservation information 2024-01-10 17:52:07 DEBUG Process-1[utils:34]: Successfully made request after 1 attempts 2024-01-10 17:52:07 DEBUG Process-1[checkin_scheduler:89]: Successfully retrieved reservation information 2024-01-10 17:52:07 DEBUG Process-1[checkin_scheduler:57]: 2 flights found under current reservation 2024-01-10 17:52:07 DEBUG Process-1[checkin_scheduler:43]: 2 total flights were found 2024-01-10 17:52:07 DEBUG Process-1[checkin_scheduler:100]: 2 new flights found 2024-01-10 17:52:07 DEBUG Process-1[checkin_scheduler:104]: Scheduling 2 flights for check-in 2024-01-10 17:52:07 DEBUG Process-1[checkin_handler:41]: Scheduling check-in for current flight 2024-01-10 17:52:07 DEBUG Process-1[checkin_handler:41]: Scheduling check-in for current flight 2024-01-10 17:52:07 DEBUG Process-1:1[checkin_handler:81]: Check-in time has passed. Going straight to check-in 2024-01-10 17:52:07 DEBUG Process-1:1[checkin_handler:124]: Attempting to check in Checking in to flight from 'Minneapolis/St. Paul (Terminal 2)' to 'Denver' for John Doe 2024-01-10 17:52:07 DEBUG Process-1:1[checkin_handler:138]: Making GET request to check in 2024-01-10 17:52:07 DEBUG Process-1[notification_handler:66]: Sending new flights notification Successfully scheduled the following flights to check in for John Doe: Flight from Minneapolis/St. Paul (Terminal 2) to Denver at 2024-01-11 12:40:00 UTC Flight from Denver to Minneapolis/St. Paul (Terminal 2) at 2024-01-15 13:50:00 UTC 2024-01-10 17:52:07 DEBUG Process-1:2[checkin_handler:89]: Sleeping until thirty minutes before check-in... 2024-01-10 17:52:09 DEBUG Process-1[checkin_scheduler:116]: 2 flights are currently scheduled. Removing old flights 2024-01-10 17:52:09 DEBUG Process-1[checkin_scheduler:132]: Successfully removed old flights. 2 flights are now scheduled 2024-01-10 17:52:09 DEBUG Process-1[reservation_monitor:93]: Checking fares for 2 flights 2024-01-10 17:52:09 DEBUG Process-1[fare_checker:30]: Checking current price for flight 2024-01-10 17:52:09 DEBUG Process-1[fare_checker:88]: Fetching reservation information 2024-01-10 17:52:11 DEBUG Process-1[utils:34]: Successfully made request after 1 attempts 2024-01-10 17:52:11 DEBUG Process-1[fare_checker:103]: Retrieving search information for the current flight 2024-01-10 17:52:12 DEBUG Process-1[utils:34]: Successfully made request after 1 attempts 2024-01-10 17:52:12 DEBUG Process-1[fare_checker:82]: Retrieving matching flights 2024-01-10 17:52:17 DEBUG Process-1[utils:41]: Failed to make request after 7 attempts: Too Many Requests 429 2024-01-10 17:52:17 DEBUG Process-1[utils:44]: Response body: { "code": 429999999, "message": "Error.", "messageKey": "ERROR", "httpStatusCode": "BAD_REQUEST", "requestId": "", "infoList": [] } 2024-01-10 17:52:17 ERROR Process-1[reservation_monitor:102]: Requesting error during fare check. Too Many Requests 429. Skipping... 2024-01-10 17:52:17 DEBUG Process-1[fare_checker:30]: Checking current price for flight 2024-01-10 17:52:17 DEBUG Process-1[fare_checker:88]: Fetching reservation information 2024-01-10 17:52:19 DEBUG Process-1[utils:34]: Successfully made request after 1 attempts 2024-01-10 17:52:19 DEBUG Process-1[fare_checker:103]: Retrieving search information for the current flight 2024-01-10 17:52:21 DEBUG Process-1[utils:34]: Successfully made request after 1 attempts 2024-01-10 17:52:21 DEBUG Process-1[fare_checker:82]: Retrieving matching flights 2024-01-10 17:52:22 DEBUG Process-1:1[utils:41]: Failed to make request after 20 attempts: Too Many Requests 429 2024-01-10 17:52:22 DEBUG Process-1:1[utils:44]: Response body: { "code": 429999999, "message": "Error.", "messageKey": "ERROR", "httpStatusCode": "BAD_REQUEST", "requestId": "", "infoList": [] } 2024-01-10 17:52:22 DEBUG Process-1:1[checkin_handler:147]: Failed to check in. Error: Too Many Requests 429. Exiting 2024-01-10 17:52:22 DEBUG Process-1:1[notification_handler:108]: Sending failed check-in notification... Failed to check in to flight XXXXX for John Doe. Reason: Too Many Requests 429. Check in at this url: https://mobile.southwest.com/check-in ```

westeras commented 10 months ago

I just started using the application this morning, but having the same issue attempting to run as a docker daemon on my home server. I was able to run (via docker compose) in the foreground and successfully check in, only when deploying as a daemon I hit the 429.

StevenMassaro commented 10 months ago

I just started using the application this morning, but having the same issue attempting to run as a docker daemon on my home server. I was able to run (via docker compose) in the foreground and successfully check in, only when deploying as a daemon I hit the 429.

I have the same experience. I get errors with docker compose up -d but not with docker compose up. Very interesting.

jdholtz commented 10 months ago

I tried v7.1 on two machines I have (each on different networks) and none of them hit a 429. I also tried with the docker-compose example from the Readme using the -d flag and still did not hit a 429.

I did print out the response headers on the failing requests and unfortunately there is no key telling you when the rate limit resets.

It doesn't seem to be rate limiting because, as you said later, adding a delay between requests does not circumvent the issue.

Could someone paste the headers they are using? Maybe there is something interesting with them that tells Southwest you are a bot. Add print(self.headers) to line 83 of lib/fare_checker.py to print right before the 429 is reached.

sevenlayercookie commented 10 months ago

Could someone paste the headers they are using? Maybe there is something interesting with them that tells Southwest you are a bot. Add print(self.headers) to line 83 of lib/fare_checker.py to print right before the 429 is reached.

Headers

``` Successfully scheduled the following flights to check in for John Doe: Flight from Minneapolis/St. Paul (Terminal 2) to Denver at 2024-01-11 12:40:00 UTC Flight from Denver to Minneapolis/St. Paul (Terminal 2) at 2024-01-15 13:50:00 UTC {'EE30zvQLWf-a': 'OMzP=bVBpVl==pHZJSY5Aur=WXFIIpSsZNvic1u317zXHPrGgNpvbCKt9pzHtU-Z8tht=WZBRrzw0_YQA0UqkQX2eC-d_ahozIL6tfCjElZyoTpBZmHyaoDEBMwkedbouLHB4deSDkfW40MOIDXr1z29l=KelDbwZVHOtqjrX0DTVMYQV5UT73gP0ozwL_qJqlXPTuRFMe13QUvmPIfFM1wzbnff5jnL5FtY_zfdR9z=W5R9X6dAe2ZKUmQ52e=Okuo6QsV24ylsNcngEeJwy9F4mlEhjnHgdF1Yt0JhaUvyzP3cWiYhJLEzejqyDZ3FPql1SSlooFWZ43-NA8R1_w_5gpdk8mEng7IMOna8kPNnV6mkc9cA5=Bf1NbgQ5se_Wt9dPXBu4ihn6Z2GtwybPhnzuiRSHFd9vJ=0MK-yp18HTAjDtrChruiFTQ692ctNSnys1I0Uo9Dnk9h2nLe46lOvusBlUhurnWcBJfRGTuGIsvcTjBHXL3SlJNGW7PN_6226fLBwRZ0TCPaYljSN5CQzZPRfJhUXBhfuSW-LCaD12Ua03MSAUkVCzfKzMC3P8q2-=8H1qatbdp6tm--gUh-aRGih9RVCMs1D3JdTtafD3FTCzs6M_9Ppz=OkqqE2ReqXKAlFEr=R9THa8B1iyEztVB70ch_dXDSwUvhnWPa8kN8XiYZK6j9LUXD3BMzAX9rRPp0jachDyPSf3A3o4jlBd1GhoukWrnFUqGyF0ZYsTa0rcwjvIHVTCFj0VyGsBMvPPhdUDBMrohdtrCNaaww52rUhrhbuD8T_G9oF87ejXyN0Q_huDBOLcpOFYUqikDo_QhQfs9m6YVPg_-S1v3vGvdZh_ZimCe-4-cv5=2UKMwh5V7Te0oDBv0bVMGv9QAyyQ4G-Up=GO6p5XnzBXXl1ygpvz1EBKnY-jXwZdYoOGnFijcWEhq92h22ZP0fLYkZy4rNuHGz5QRoIiJXkPosDtmFmySHBuvcWzd0aP8eBmhqARr7HE1iYFan-BzkX_7YzFDuIEyrlr04w0UGE9kb-9MtiBB5WKyRQSnlak1SzSL-L8IX=uLlHhNWd8Je6JJ7c6EclcKd3vNqCvsM93oE9AkeyYBi_=O0wkmKz1CNmGWWQdcurjCUV5D-m923OeQErOnDUG0XtDUY7iBw2yPjv9o_6kp5ZJ4XnEiRXoViOIb2ce7qimjTYNGwmJcbKErTaGCjN-u4DrGoiw-KIzE7Xqk1gZQAqOBCQn-RJXLtsE3ov9rlsrzuF7Thd36=I0RQr6bWiqfUPrFLriaCvr6ZyAw51h396coUNBcF9Low0V_YF0qYCHuzwF7uI7vzlAH4gsBtS_XcvXhdVJ7lYjkUwoCkQoPiYhEOYkLV3cfzZaYjOREJu1247=I1YHYVowTTEghLpFl7r=vo1N=fd3Lwc-=2O74kepkFTh5-QfoW0bL0c4KO2yXDaQnKztO4cP87eGsywMagO-igwcXHavqLVdeK-_wCzmSlfDZ2Y6GsrX1E83gwA=H4w_Vgnz=gA=5kpC4LdOpNDJM_IMnIP9ECzwo_9o3U6KkFiD5OhN4kJt6Ubh-2s4rS0YHj_jFOiKeN6KgKWd1-DImYwNDuyEiIg20WjjSTSZoTVzdc8o7VeFR2rstsBzeOX3JF8zq8_5uoyPGTo5esP58TqLzm5G_O436F93J92yLmFDvBAiX8zwfQdMz-BR6M9UvOZGP9lqkzyuAUcOB6EfFHl7tD-lUrPglSBo98AKNYya92cADSXL1OJ5QC=47RPMFu6Ver2onOa_2B6X8JRiK76mfELaGg6NOkIJyDUSm4qWmLrfL1_Jviq8zu9N3ROHKYIukrL8k6GuV2XBolRRhSllFhdl5rvvILBqEGMhgyk1BLByIs9G4sg2H-PwTQ1YX4f2Mvv=bJbNKozFvpoPKz924cRZ6Iu9Dw5d5-874W8izM1mvlZPnfEGNLG0VBQF-kW1mFZYEZDQua_caMGpdHZS8WHeR0rSj65btofVGjpnUC3McQwyLt_bEMw_ujMlaqjZtPCy3u1Fo5fnWayp7bUfrV_vTNPln7fZWMBvevScMTw-gIeukdKOz6j8_p4yRHLUTkv_7wRfYGnrGVLO59Cwf19vesBWIidSLmoEnGvyRfa_Qcq5ifYK4hJh43=A69fZOHzjvUykd41UU=MoEMb=mF9odHVFRIoZKl6hJ50N8os33Af7an-=c5DzF7OZ=XwNCmrhWNy7R2ulhKKvjz3O2fpKt6e2VtDUmUESnQ2UOkAMmpWgBcA8pP-A0JmTj93oeCv2R76m7gmSPT4PrVFPS67GbVPkCT0haWqwG0AtwZ89qkGbmIKEjfXCpSUk-UiKhwdaab5irB1M82w0vWuUtiugrf4HUvdN41=pruzBv3iMe9E2bYttjieAM4G3OCOO=en1X0KyD1ll9sX8lb9X-3SiiTumdr6cnI532dBVuI203gAR1bHqzhQDTgkB3nJ9gyb5L1_Qw4Kcciw1W8wgUXwtOjRLzocSUHvACGZQK-qC0EKgU8-270d9sBWsSsBdbH=Dp9aMMAp3IADkNsDOL4Slban6ordH4ym9hGjYiHGRrXkA1FOSzTifPWE727dOp=LTbNb3CdiMk-z8-Hoa274qtTDfIFg=78mkngP8=XILAWcZAQvNdW2Ui2stNAdfrCC=XbsXf8J_tgDFSFaSEBv_beMY3v9rPW8DVoZ8s7BPW_TpJgwr1PV09Gu0YASm9RP4Wh_vNqOcMdtzCdJiDy7fXrmNEQhI-OkyNVNRKiGaiZRdAZuadUU2WsPfoFViv=AEo5KCgi540_bkACaYLB1DZyCN0OqYKLa9A0QaOJzQCZApO9KowS_c=0lwr2asYY8n=TSJ7_gQLY4LAlvdWp2A6GAw_yZTwuGO0VpEqFqacAYUTTMFdWOARG5oNrNDboIsTnSUfvlESFY8Wmf24IqKX8e1JdeQzjTZdDC5-_IN8tLjmMcBzXop6kNdqrTjbZBN9qk9ISJd=O8rB4FowjfvEsvz=TcGsOr619O0p2PmAiwV7UKjfyL11VtyJ4DrEj7ErHVo-nJtDt5kI02WupVnNYYOag_vNNp5cMrMCsZcFz4_6yNdw', 'EE30zvQLWf-b': '-a7d0wz', 'EE30zvQLWf-c': 'AKAkTfaMAQAAKPb_paP3sTUVsqAe3dippV6vvvHv0JcRgYORc_8V9nJxB7IK', 'EE30zvQLWf-d': 'ABaAhIDBCKGFgQGAAYIQgISigaIAwBGAzvpizi_33wf_FfZycQeyCgAAAABnMdsJAA2Ji2BO8r6vOfHqt_JGKio', 'EE30zvQLWf-f': 'AzMlUfaMAQAAQKg7vsgh-SDXVYO_RGDea01z_IXGZRUIWRtIRibM2XfY8j1vAZh1c9iucgHcwH8AADQwAAAAAA==', 'EE30zvQLWf-z': 'q', 'User-Agent': 'Mozilla/5.0 (X11; Linux aarch64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.5938.62 Safari/537.36', 'X-API-Key': 'l7xx2c186c1297274b828b1872e22edfe67a', 'X-Channel-ID': 'MWEB'} Requesting error during fare check. Too Many Requests 429. Skipping... {'EE30zvQLWf-a': 'OMzP=bVBpVl==pHZJSY5Aur=WXFIIpSsZNvic1u317zXHPrGgNpvbCKt9pzHtU-Z8tht=WZBRrzw0_YQA0UqkQX2eC-d_ahozIL6tfCjElZyoTpBZmHyaoDEBMwkedbouLHB4deSDkfW40MOIDXr1z29l=KelDbwZVHOtqjrX0DTVMYQV5UT73gP0ozwL_qJqlXPTuRFMe13QUvmPIfFM1wzbnff5jnL5FtY_zfdR9z=W5R9X6dAe2ZKUmQ52e=Okuo6QsV24ylsNcngEeJwy9F4mlEhjnHgdF1Yt0JhaUvyzP3cWiYhJLEzejqyDZ3FPql1SSlooFWZ43-NA8R1_w_5gpdk8mEng7IMOna8kPNnV6mkc9cA5=Bf1NbgQ5se_Wt9dPXBu4ihn6Z2GtwybPhnzuiRSHFd9vJ=0MK-yp18HTAjDtrChruiFTQ692ctNSnys1I0Uo9Dnk9h2nLe46lOvusBlUhurnWcBJfRGTuGIsvcTjBHXL3SlJNGW7PN_6226fLBwRZ0TCPaYljSN5CQzZPRfJhUXBhfuSW-LCaD12Ua03MSAUkVCzfKzMC3P8q2-=8H1qatbdp6tm--gUh-aRGih9RVCMs1D3JdTtafD3FTCzs6M_9Ppz=OkqqE2ReqXKAlFEr=R9THa8B1iyEztVB70ch_dXDSwUvhnWPa8kN8XiYZK6j9LUXD3BMzAX9rRPp0jachDyPSf3A3o4jlBd1GhoukWrnFUqGyF0ZYsTa0rcwjvIHVTCFj0VyGsBMvPPhdUDBMrohdtrCNaaww52rUhrhbuD8T_G9oF87ejXyN0Q_huDBOLcpOFYUqikDo_QhQfs9m6YVPg_-S1v3vGvdZh_ZimCe-4-cv5=2UKMwh5V7Te0oDBv0bVMGv9QAyyQ4G-Up=GO6p5XnzBXXl1ygpvz1EBKnY-jXwZdYoOGnFijcWEhq92h22ZP0fLYkZy4rNuHGz5QRoIiJXkPosDtmFmySHBuvcWzd0aP8eBmhqARr7HE1iYFan-BzkX_7YzFDuIEyrlr04w0UGE9kb-9MtiBB5WKyRQSnlak1SzSL-L8IX=uLlHhNWd8Je6JJ7c6EclcKd3vNqCvsM93oE9AkeyYBi_=O0wkmKz1CNmGWWQdcurjCUV5D-m923OeQErOnDUG0XtDUY7iBw2yPjv9o_6kp5ZJ4XnEiRXoViOIb2ce7qimjTYNGwmJcbKErTaGCjN-u4DrGoiw-KIzE7Xqk1gZQAqOBCQn-RJXLtsE3ov9rlsrzuF7Thd36=I0RQr6bWiqfUPrFLriaCvr6ZyAw51h396coUNBcF9Low0V_YF0qYCHuzwF7uI7vzlAH4gsBtS_XcvXhdVJ7lYjkUwoCkQoPiYhEOYkLV3cfzZaYjOREJu1247=I1YHYVowTTEghLpFl7r=vo1N=fd3Lwc-=2O74kepkFTh5-QfoW0bL0c4KO2yXDaQnKztO4cP87eGsywMagO-igwcXHavqLVdeK-_wCzmSlfDZ2Y6GsrX1E83gwA=H4w_Vgnz=gA=5kpC4LdOpNDJM_IMnIP9ECzwo_9o3U6KkFiD5OhN4kJt6Ubh-2s4rS0YHj_jFOiKeN6KgKWd1-DImYwNDuyEiIg20WjjSTSZoTVzdc8o7VeFR2rstsBzeOX3JF8zq8_5uoyPGTo5esP58TqLzm5G_O436F93J92yLmFDvBAiX8zwfQdMz-BR6M9UvOZGP9lqkzyuAUcOB6EfFHl7tD-lUrPglSBo98AKNYya92cADSXL1OJ5QC=47RPMFu6Ver2onOa_2B6X8JRiK76mfELaGg6NOkIJyDUSm4qWmLrfL1_Jviq8zu9N3ROHKYIukrL8k6GuV2XBolRRhSllFhdl5rvvILBqEGMhgyk1BLByIs9G4sg2H-PwTQ1YX4f2Mvv=bJbNKozFvpoPKz924cRZ6Iu9Dw5d5-874W8izM1mvlZPnfEGNLG0VBQF-kW1mFZYEZDQua_caMGpdHZS8WHeR0rSj65btofVGjpnUC3McQwyLt_bEMw_ujMlaqjZtPCy3u1Fo5fnWayp7bUfrV_vTNPln7fZWMBvevScMTw-gIeukdKOz6j8_p4yRHLUTkv_7wRfYGnrGVLO59Cwf19vesBWIidSLmoEnGvyRfa_Qcq5ifYK4hJh43=A69fZOHzjvUykd41UU=MoEMb=mF9odHVFRIoZKl6hJ50N8os33Af7an-=c5DzF7OZ=XwNCmrhWNy7R2ulhKKvjz3O2fpKt6e2VtDUmUESnQ2UOkAMmpWgBcA8pP-A0JmTj93oeCv2R76m7gmSPT4PrVFPS67GbVPkCT0haWqwG0AtwZ89qkGbmIKEjfXCpSUk-UiKhwdaab5irB1M82w0vWuUtiugrf4HUvdN41=pruzBv3iMe9E2bYttjieAM4G3OCOO=en1X0KyD1ll9sX8lb9X-3SiiTumdr6cnI532dBVuI203gAR1bHqzhQDTgkB3nJ9gyb5L1_Qw4Kcciw1W8wgUXwtOjRLzocSUHvACGZQK-qC0EKgU8-270d9sBWsSsBdbH=Dp9aMMAp3IADkNsDOL4Slban6ordH4ym9hGjYiHGRrXkA1FOSzTifPWE727dOp=LTbNb3CdiMk-z8-Hoa274qtTDfIFg=78mkngP8=XILAWcZAQvNdW2Ui2stNAdfrCC=XbsXf8J_tgDFSFaSEBv_beMY3v9rPW8DVoZ8s7BPW_TpJgwr1PV09Gu0YASm9RP4Wh_vNqOcMdtzCdJiDy7fXrmNEQhI-OkyNVNRKiGaiZRdAZuadUU2WsPfoFViv=AEo5KCgi540_bkACaYLB1DZyCN0OqYKLa9A0QaOJzQCZApO9KowS_c=0lwr2asYY8n=TSJ7_gQLY4LAlvdWp2A6GAw_yZTwuGO0VpEqFqacAYUTTMFdWOARG5oNrNDboIsTnSUfvlESFY8Wmf24IqKX8e1JdeQzjTZdDC5-_IN8tLjmMcBzXop6kNdqrTjbZBN9qk9ISJd=O8rB4FowjfvEsvz=TcGsOr619O0p2PmAiwV7UKjfyL11VtyJ4DrEj7ErHVo-nJtDt5kI02WupVnNYYOag_vNNp5cMrMCsZcFz4_6yNdw', 'EE30zvQLWf-b': '-a7d0wz', 'EE30zvQLWf-c': 'AKAkTfaMAQAAKPb_paP3sTUVsqAe3dippV6vvvHv0JcRgYORc_8V9nJxB7IK', 'EE30zvQLWf-d': 'ABaAhIDBCKGFgQGAAYIQgISigaIAwBGAzvpizi_33wf_FfZycQeyCgAAAABnMdsJAA2Ji2BO8r6vOfHqt_JGKio', 'EE30zvQLWf-f': 'AzMlUfaMAQAAQKg7vsgh-SDXVYO_RGDea01z_IXGZRUIWRtIRibM2XfY8j1vAZh1c9iucgHcwH8AADQwAAAAAA==', 'EE30zvQLWf-z': 'q', 'User-Agent': 'Mozilla/5.0 (X11; Linux aarch64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.5938.62 Safari/537.36', 'X-API-Key': 'l7xx2c186c1297274b828b1872e22edfe67a', 'X-Channel-ID': 'MWEB'} Failed to check in to flight XXXXXX for John Doe. Reason: Too Many Requests 429. Check in at this url: https://mobile.southwest.com/check-in Requesting error during fare check. Too Many Requests 429. Skipping... ```

jwin3 commented 10 months ago

I got a "Reason: Too Many Requests 429" when it went to try to check me in yesterday. It happened for me and another person for a different flight. I'm on 7.1 as well. I only have price checking interval at every 12 hours, so I don't think I triggered a southwest block or anything. No VPN either.

jdholtz commented 10 months ago

Thanks @sevenlayercookie. Doesn’t seem like anything out of the ordinary.

Is anyone who is getting this error running this with python or is everyone running it in Docker?

cdshann commented 10 months ago

I just got a 429 trying to check-in for my flight this morning. I'm running the script with just python and not using Docker running on an EC2 instance in tmux. Before the script attempted to check me in I also saw the 429s for the fare checks.

Failed to check in to flight ... for ... Reason: Too Many Requests 429.
Check in at this url: https://mobile.southwest.com/check-in
jdholtz commented 10 months ago

I can now reproduce this issue while running in Docker (even with the -d flag). However, not running it directly with Python though.

jwin3 commented 10 months ago

I was able to check in to some flights this morning without any issue. Same version I was on previously with no changes. Using Docker image on my qnap nas.

jdholtz commented 10 months ago

I was able to check in to some flights this morning without any issue

Did some flights still fail to check in with the 429?

jwin3 commented 10 months ago

I was able to check in to some flights this morning without any issue

Did some flights still fail to check in with the 429?

Not today, my flight and another person (2 different accounts and flight times) on 1/10/24 during check-in I got a 429. Today, I had no issues checking in.

StevenMassaro commented 10 months ago

I was able to check in to some flights this morning without any issue

Did some flights still fail to check in with the 429?

I got one 429 today during the fare check, with seven successful fare checks.

conor888 commented 10 months ago

I got a 429 today on a flight check-in, I can try restarting the docker container and seeing if that fixes it for the returning flight in a few days. Would turning off the fare check help any? Right now mine's set to the default 24 hours.

jdholtz commented 10 months ago

Would turning off the fare check help any?

Unfortunately not. I will look to see if I can fix this issue soon.

wpkato commented 10 months ago

FYI...Getting same error on flight checkin today. Successfully worked month of Dec '23.

Logs Successfully scheduled the following flights to check in for YYYYY: Flight from Chicago (Midway) to New York (LaGuardia) at 2024-01-19 14:00:00 UTC Flight from New York (LaGuardia) to Chicago (Midway) at 2024-01-22 22:20:00 UTC 2024-01-18 17:00:47 DEBUG Process-1:2[checkin_handler:89]: Sleeping until thirty minutes before check-in... 2024-01-18 17:00:47 DEBUG Process-1[checkin_scheduler:116]: 2 flights are currently scheduled. Removing old flights 2024-01-18 17:00:47 DEBUG Process-1[checkin_scheduler:132]: Successfully removed old flights. 2 flights are now scheduled 2024-01-18 17:00:47 DEBUG Process-1[reservation_monitor:93]: Checking fares for 2 flights 2024-01-18 17:00:47 DEBUG Process-1[fare_checker:30]: Checking current price for flight 2024-01-18 17:00:47 DEBUG Process-1[fare_checker:88]: Fetching reservation information 2024-01-18 17:00:52 DEBUG Process-1[utils:34]: Successfully made request after 1 attempts 2024-01-18 17:00:52 DEBUG Process-1[fare_checker:103]: Retrieving search information for the current flight 2024-01-18 17:00:52 DEBUG Process-1[reservation_monitor:104]: Flight cannot be changed online. Skipping fare check 2024-01-18 17:00:52 DEBUG Process-1[fare_checker:30]: Checking current price for flight 2024-01-18 17:00:52 DEBUG Process-1[fare_checker:88]: Fetching reservation information 2024-01-18 17:00:56 DEBUG Process-1[utils:34]: Successfully made request after 1 attempts 2024-01-18 17:00:56 DEBUG Process-1[fare_checker:103]: Retrieving search information for the current flight 2024-01-18 17:00:56 DEBUG Process-1[reservation_monitor:104]: Flight cannot be changed online. Skipping fare check 2024-01-18 17:00:56 DEBUG Process-1[reservation_monitor:167]: Lock released 2024-01-18 17:00:56 DEBUG Process-1[reservation_monitor:116]: Sleeping for 86352 seconds 2024-01-18 17:00:59 DEBUG Process-1:1[utils:41]: Failed to make request after 20 attempts: Too Many Requests 429 2024-01-18 17:00:59 DEBUG Process-1:1[utils:44]: Response body: { "code": 429999999, "message": "Error.", "messageKey": "ERROR", "httpStatusCode": "BAD_REQUEST", "requestId": "", "infoList": [] } 2024-01-18 17:00:59 DEBUG Process-1:1[checkin_handler:147]: Failed to check in. Error: Too Many Requests 429. Exiting Failed to check in to flight XXXX for YYYYY. Reason: Too Many Requests 429. Check in at this url: https://mobile.southwest.com/check-in 2024-01-18 17:00:59 DEBUG Process-1:1[notification_handler:108]: Sending failed check-in notification...
jdholtz commented 10 months ago

Thanks for the logs. I haven't had much time to look into this right now, but I can later this afternoon. Currently, it seems like people are having issues in Docker but not when running it with Python (at least, this is my case). The logs and headers are not anything outside of the ordinary. Something with Docker is probably making Southwest detect the bot (not sure what though).

jdholtz commented 10 months ago

I've done some debugging. Unfortunately, I haven't come closer to finding the issue and am not sure how to move forward, but I will detail what I have done so far below. It would be great if other people can help debug so we can resolve this.

Most likely not the issue:

  1. The code. It works perfectly fine for me running in python. Additionally, in the Docker container, running the same exact post request for the fare checker (same query and headers) sometimes gives a 200 but mostly a 429.
  2. The docker/containerd version. The Docker engine hasn't updated since December and Containerd's last update before this issue started happening was also December
  3. A specific network configuration. Many people can reproduce the issue on many different networks

Potentially the issue:

  1. Southwest's website changed to either improve their bot detection or just randomly fail occasionally? I doubt it as it works outside of Python and works haphazardly within Docker, but could be possible.
  2. How requests are sent/networking through a Docker container. Perhaps there is some revealing information that indicates to the Southwest server it is not a human. I did inspect two requests (one 200 and one 429) and they didn't seem to reveal anything special as to what the cause of the 429 is.

These are only the situations I have looked into. Let me know if you guys think of anything else to try/debug it yourselves.

cdshann commented 10 months ago

Just to highlight an earlier data point - my previous comment called out a failure from just running the script directly in Python. No Docker involved. I had both legs of my checkin fail with a 429 (same bad request message too).

I did run this from an AWS EC2 instance so it is possible they flagged my EIP as a potential bot.

jdholtz commented 10 months ago

my previous comment called out a failure from just running the script directly in Python. No Docker involved

Ah, thanks for pointing that out. I doubt they flagged the EIP because many other people are getting the same issue and are running it from their home network.

cdshann commented 10 months ago

Makes sense. I actually just (5 minutes ago) got this failure again after using tailscale to funnel traffic from the EC2 instance back over my home ISP.

wpkato commented 10 months ago

FYI, I just tried to run autocheckin outside of my docker/portainer instance and was able to get it to login to SWA and schedule my flight next week with no error messages. It even showed the "successful" seat obtained for my flight tomorrow that I had to get manually. I'm hoping it will execute the autocheckin for my flight next week but I will try and monitor it.

I did remember I updated containerd.io earlier this week on my debian server, through an apt update/upgrade but not sure if that has anything to do with the issue.

jdholtz commented 10 months ago

was able to get it to login to SWA and schedule my flight next week with no error messages

It should definitely work with logging in and scheduling flights. The issue is that it is running into a 429 during a POST request. This happens in 2 areas:

  1. When checking fares (to retrieve matching flights)
  2. The second part of the check-in

However, GET requests do go through perfectly (at least, I haven't had a single one fail).

wpkato commented 10 months ago

Good to know. It did go through the check fares routine with no errors. Looks like it did a POST request successfully.

2024-01-18 21:23:04 DEBUG Process-1:2[checkin_handler:89]: Sleeping until thirty minutes before check-in... 2024-01-18 21:23:04 DEBUG Process-1[checkin_scheduler:116]: 2 flights are currently scheduled. Removing old flights 2024-01-18 21:23:04 DEBUG Process-1[checkin_scheduler:132]: Successfully removed old flights. 2 flights are now scheduled 2024-01-18 21:23:04 DEBUG Process-1[reservation_monitor:93]: Checking fares for 2 flights 2024-01-18 21:23:04 DEBUG Process-1[fare_checker:30]: Checking current price for flight 2024-01-18 21:23:04 DEBUG Process-1[fare_checker:88]: Fetching reservation information 2024-01-18 21:23:07 DEBUG Process-1:1[utils:34]: Successfully made request after 1 attempts 2024-01-18 21:23:07 DEBUG Process-1:1[checkin_handler:144]: Making POST request to check in 2024-01-18 21:23:08 DEBUG Process-1[utils:34]: Successfully made request after 1 attempts 2024-01-18 21:23:08 DEBUG Process-1[fare_checker:103]: Retrieving search information for the current flight 2024-01-18 21:23:08 DEBUG Process-1[reservation_monitor:104]: Flight cannot be changed online. Skipping fare check 2024-01-18 21:23:08 DEBUG Process-1[fare_checker:30]: Checking current price for flight 2024-01-18 21:23:08 DEBUG Process-1[fare_checker:88]: Fetching reservation information 2024-01-18 21:23:08 DEBUG Process-1:1[utils:34]: Successfully made request after 1 attempts 2024-01-18 21:23:08 DEBUG Process-1:1[checkin_handler:151]: Successfully checked in! 2024-01-18 21:23:08 DEBUG Process-1:1[notification_handler:99]: Sending successful check-in notification... Successfully checked in to flight from 'Chicago (Midway)' to 'New York (LaGuardia)' for YYYYY! YYYYY got JJJJ!

Sorry, I'm not of more help.

So what you're saying is that even though I'm not getting the same errors that were appearing while running the script in docker, it still may fail once it has to actually do the seat checkin? Thanks for your help

jdholtz commented 10 months ago

So what you're saying is that even though I'm not getting the same errors that were appearing while running the script in docker, it still may fail once it has to actually do the seat checkin? Thanks for your help

The POST requests do sometimes go through (seemingly arbitrarily) in Docker. However, I have not run into a 429 outside of a Docker container yet. If you run the script outside of Docker a few times without it hitting a 429, it should check you in correctly whenever the time comes.

jdholtz commented 10 months ago

I did have a successful fare check and flight check-in yesterday using Docker. I also tried today and was not getting a 429 (more on that below). Are other people still running into this issue?

I noticed that the POST request of the check-in may sometimes fail a few times with a 429 before succeeding. For people who are still running into the issue, does changing line 83 of lib/fare_checker.py to response = make_request("POST", site, self.headers, query) (removing the max_attempts parameter so it defaults to 20) make it work?

wpkato commented 10 months ago

I had a successful fare-checks and check-in to my flight tomorrow. Outside of docker. Seems confirm your suspicion that it's not the code, but the code with docker.

On Sat, Jan 20, 2024, 19:52 Joey Holtzman @.***> wrote:

I did have a successful fare check and flight check-in yesterday using Docker. I also tried today and was not getting a 429 (more on that below). Are other people still running into this issue?

I noticed that the POST request of the check-in may sometimes fail a few times with a 429 before succeeding. For people who are still running into the issue, does changing line 83 of lib/fare_checker.py to response = make_request("POST", site, self.headers, query) (removing the max_attempts parameter so it defaults to 20) make it work?

— Reply to this email directly, view it on GitHub https://github.com/jdholtz/auto-southwest-check-in/issues/201#issuecomment-1902465477, or unsubscribe https://github.com/notifications/unsubscribe-auth/A2QJTU7LO3ORB5YEH3VRQ23YPRRFHAVCNFSM6AAAAABBTESKY6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBSGQ3DKNBXG4 . You are receiving this because you commented.Message ID: @.***>

jdholtz commented 10 months ago

Seems confirm your suspicion that it's not the code, but the code with docker.

Actually, my results were from running it in Docker. Again today, I had another successful check-in and fare check within Docker.

bradzab0623 commented 10 months ago

I've been seeing this issue as well - I am running inside docker. I upped the retry attempts to 20, but it seems it either succeeds on the first or second, or it fails after 20. I haven't seen a success after more than 2 attempts. I've also tried sleeping 30 seconds in between attempts, same thing.

The success rate seems to be pretty low for me - ~20%

Running outside of docker seems to work fine.

StevenMassaro commented 10 months ago

I continue to regularly have 429's through Docker when doing the fare check, as of this morning.

ntalekt commented 10 months ago

Failed to check-in for flight today. Just restarted the container, logs below.

/app/logs # cat auto-southwest-check-in.log
2024-01-23 21:18:19 DEBUG MainProcess[log:23]: Initialized the application
2024-01-23 21:18:19 DEBUG MainProcess[main:112]: Auto-Southwest Check-In v7.1
2024-01-23 21:18:19 DEBUG MainProcess[main:113]: Called with 0 arguments
2024-01-23 21:18:19 DEBUG MainProcess[config:107]: Initializing configuration file
2024-01-23 21:18:19 DEBUG MainProcess[config:135]: Reading the configuration file
2024-01-23 21:18:19 DEBUG MainProcess[config:53]: Setting check fares to False
2024-01-23 21:18:19 DEBUG MainProcess[config:67]: Setting notification level to <NotificationLevel.INFO: 1>
2024-01-23 21:18:19 DEBUG MainProcess[config:80]: Using 1 notification services
2024-01-23 21:18:19 DEBUG MainProcess[config:84]: Setting retrieval interval to 24 hours
2024-01-23 21:18:19 DEBUG MainProcess[config:118]: Creating configurations for 0 accounts
2024-01-23 21:18:19 DEBUG MainProcess[config:125]: Creating configurations for 1 reservations
2024-01-23 21:18:19 DEBUG MainProcess[main:142]: Monitoring 0 accounts and 1 reservations
2024-01-23 21:18:20 DEBUG Process-1[reservation_monitor:60]: Acquiring lock...
2024-01-23 21:18:20 DEBUG Process-1[reservation_monitor:62]: Lock acquired
2024-01-23 21:18:20 DEBUG Process-1[checkin_scheduler:50]: Refreshing headers for current session
2024-01-23 21:18:20 DEBUG Process-1[webdriver:103]: Starting webdriver for current session
2024-01-23 21:18:23 DEBUG Process-1[webdriver:119]: Using browser version: 117.0.5938.62
2024-01-23 21:18:23 DEBUG Process-1[webdriver:123]: Loading Southwest Check-In page
2024-01-23 21:18:31 DEBUG Process-1[webdriver:65]: Waiting for valid headers
2024-01-23 21:18:31 DEBUG Process-1[webdriver:152]: Waiting for headers_set to be set
2024-01-23 21:18:32 DEBUG Process-1[webdriver:156]: headers_set set successfully
2024-01-23 21:18:32 DEBUG Process-1[reservation_monitor:84]: Scheduling flight check-ins for 1 reservations
2024-01-23 21:18:32 DEBUG Process-1[checkin_scheduler:76]: Retrieving reservation information
2024-01-23 21:18:36 DEBUG Process-1[utils:34]: Successfully made request after 1 attempts
2024-01-23 21:18:36 DEBUG Process-1[checkin_scheduler:89]: Successfully retrieved reservation information
2024-01-23 21:18:36 DEBUG Process-1[checkin_scheduler:57]: 2 flights found under current reservation
2024-01-23 21:18:36 DEBUG Process-1[checkin_scheduler:43]: 2 total flights were found
2024-01-23 21:18:36 DEBUG Process-1[checkin_scheduler:100]: 2 new flights found
2024-01-23 21:18:36 DEBUG Process-1[checkin_scheduler:104]: Scheduling 2 flights for check-in
2024-01-23 21:18:36 DEBUG Process-1[checkin_handler:41]: Scheduling check-in for current flight
2024-01-23 21:18:36 DEBUG Process-1[checkin_handler:41]: Scheduling check-in for current flight
2024-01-23 21:18:36 DEBUG Process-1:1[checkin_handler:81]: Check-in time has passed. Going straight to check-in
2024-01-23 21:18:36 DEBUG Process-1:1[checkin_handler:124]: Attempting to check in
2024-01-23 21:18:36 DEBUG Process-1:1[checkin_handler:138]: Making GET request to check in
2024-01-23 21:18:36 DEBUG Process-1[notification_handler:66]: Sending new flights notification
2024-01-23 21:18:36 DEBUG Process-1:2[checkin_handler:89]: Sleeping until thirty minutes before check-in...
2024-01-23 21:18:37 DEBUG Process-1[checkin_scheduler:116]: 2 flights are currently scheduled. Removing old flights
2024-01-23 21:18:37 DEBUG Process-1[checkin_scheduler:132]: Successfully removed old flights. 2 flights are now scheduled
2024-01-23 21:18:37 DEBUG Process-1[reservation_monitor:80]: Lock released
2024-01-23 21:18:37 DEBUG Process-1[reservation_monitor:116]: Sleeping for 86383 seconds
2024-01-23 21:18:53 DEBUG Process-1:1[utils:41]: Failed to make request after 20 attempts: Too Many Requests 429
2024-01-23 21:18:53 DEBUG Process-1:1[utils:44]: Response body: {
    "code": 429999999,
    "message": "Error.",
    "messageKey": "ERROR",
    "httpStatusCode": "BAD_REQUEST",
    "requestId": "",
    "infoList": []
   }
2024-01-23 21:18:53 DEBUG Process-1:1[checkin_handler:147]: Failed to check in. Error: Too Many Requests 429. Exiting
2024-01-23 21:18:53 DEBUG Process-1:1[notification_handler:108]: Sending failed check-in notification...

Update: I tried a few more versions and jdholtz/auto-southwest-check-in:v5.0 worked first try

/app/logs # cat auto-southwest-check-in.log
2024-01-23 22:54:15 DEBUG MainProcess[log:23]: Initialized the application
2024-01-23 22:54:15 DEBUG MainProcess[main:112]: Auto-Southwest Check-In v5.0
2024-01-23 22:54:15 DEBUG MainProcess[main:113]: Called with 0 arguments
2024-01-23 22:54:15 DEBUG MainProcess[config:110]: Initializing configuration file
2024-01-23 22:54:15 DEBUG MainProcess[config:138]: Reading the configuration file
2024-01-23 22:54:15 DEBUG MainProcess[config:58]: Setting check fares to False
2024-01-23 22:54:15 DEBUG MainProcess[config:70]: Setting notification level to <NotificationLevel.INFO: 1>
2024-01-23 22:54:15 DEBUG MainProcess[config:83]: Using 1 notification services
2024-01-23 22:54:15 DEBUG MainProcess[config:87]: Setting retrieval interval to 24 hours
2024-01-23 22:54:15 DEBUG MainProcess[config:121]: Creating configurations for 0 accounts
2024-01-23 22:54:15 DEBUG MainProcess[config:128]: Creating configurations for 1 reservations
2024-01-23 22:54:15 DEBUG MainProcess[main:142]: Monitoring 0 accounts and 1 reservations
2024-01-23 22:54:16 DEBUG Process-1[checkin_scheduler:50]: Refreshing headers for current session
2024-01-23 22:54:16 DEBUG Process-1[webdriver:138]: Starting webdriver for current session
2024-01-23 22:54:18 DEBUG Process-1[webdriver:147]: Loading Southwest Check-In page
2024-01-23 22:54:28 DEBUG Process-1[webdriver:60]: Filling out a check-in form to get valid headers
2024-01-23 22:54:30 DEBUG Process-1[webdriver:200]: Setting valid headers from previous request
2024-01-23 22:54:30 DEBUG Process-1[reservation_monitor:74]: Scheduling flight check-ins for 1 reservations
2024-01-23 22:54:30 DEBUG Process-1[checkin_scheduler:76]: Retrieving reservation information
2024-01-23 22:54:34 DEBUG Process-1[utils:34]: Successfully made request after 1 attempts
2024-01-23 22:54:34 DEBUG Process-1[checkin_scheduler:89]: Successfully retrieved reservation information
2024-01-23 22:54:34 DEBUG Process-1[checkin_scheduler:57]: 2 flights found under current reservation
2024-01-23 22:54:34 DEBUG Process-1[checkin_scheduler:43]: 2 total flights were found
2024-01-23 22:54:34 DEBUG Process-1[checkin_scheduler:100]: 2 new flights found
2024-01-23 22:54:34 DEBUG Process-1[checkin_scheduler:104]: Scheduling 2 flights for check-in
2024-01-23 22:54:34 DEBUG Process-1[checkin_handler:40]: Scheduling check-in for current flight
2024-01-23 22:54:34 DEBUG Process-1[checkin_handler:40]: Scheduling check-in for current flight
2024-01-23 22:54:34 DEBUG Process-1:1[checkin_handler:79]: Check-in time has passed. Going straight to check-in
2024-01-23 22:54:34 DEBUG Process-1:1[checkin_handler:114]: Attempting to check in
2024-01-23 22:54:34 DEBUG Process-1:1[checkin_handler:128]: Making GET request to check in
2024-01-23 22:54:34 DEBUG Process-1[notification_handler:55]: Sending new flights notification
2024-01-23 22:54:34 DEBUG Process-1:2[checkin_handler:87]: Sleeping until ten minutes before check-in...
2024-01-23 22:54:34 DEBUG Process-1[checkin_scheduler:116]: 2 flights are currently scheduled. Removing old flights
2024-01-23 22:54:34 DEBUG Process-1[checkin_scheduler:132]: Successfully removed old flights. 2 flights are now scheduled
2024-01-23 22:54:34 DEBUG Process-1[reservation_monitor:106]: Sleeping for 86382 seconds
2024-01-23 22:54:35 DEBUG Process-1:1[utils:34]: Successfully made request after 1 attempts
2024-01-23 22:54:35 DEBUG Process-1:1[checkin_handler:134]: Making POST request to check in
2024-01-23 22:54:36 DEBUG Process-1:1[utils:34]: Successfully made request after 1 attempts
2024-01-23 22:54:36 DEBUG Process-1:1[checkin_handler:141]: Successfully checked in!
2024-01-23 22:54:36 DEBUG Process-1:1[notification_handler:88]: Sending successful check-in notification...
hildebrau commented 10 months ago

I had experienced another project similar to this one that was having Header issues that caused 429 errors often. Perhaps their "fixes" will work on this project? See: https://github.com/byalextran/southwest-headers/issues/6

jdholtz commented 10 months ago

I had experienced another project similar to this one that was having Header issues that caused 429 errors often. Perhaps their "fixes" will work on this project?

@hildebrau Thanks for the suggestion. Unfortunately, I don't think it is the headers as I made requests with the same headers in two different instances and one returned a 200 and the other returned a 429 (although I'm not confident on this assertion). It may somehow be that SeleniumBase is being detected and Southwest is giving invalid headers, but undetected-chromedriver is not being detected (as it works in v5.1 below).

Update: I tried a few more versions and jdholtz/auto-southwest-check-in:v5.0 worked first try

@ntalekt good catch, this works for me too (on v5.1 even). I will investigate on this further.

hildebrau commented 10 months ago

@jdholtz, yes, the undetected-chromedriver is likely the answer.

I tried v5.1 and didn't receive any errors either. However, I also don't see it trying to check prices. Perhaps that was a later feature added. (I just discovered this particular app yesterday).

jdholtz commented 10 months ago

However, I also don't see it trying to check prices. Perhaps that was a later feature added. (I just discovered this particular app yesterday).

In v5.1, check_fares defaulted to false, so you will need to put check_fares: true in your config for the script to check flight fares. This default was changed in v6.0

hildebrau commented 10 months ago

In v5.1, check_fares defaulted to false, so you will need to put check_fares: true in your config for the script to check flight fares. This default was changed in v6.0

Strange. I do have that in my config file at the first level.. but not in the accounts or reservation level..

{ "check_fares": true, ..snip..

jdholtz commented 10 months ago

Strange. I do have that in my config file at the first level.. but not in the accounts or reservation level..

Hm, I am able to check fares just fine with that configuration option (with an account and a reservation). When that option is in your config, are there are no lines in the logs saying "Checking fares for X flights"?

jwin3 commented 10 months ago

So I'm running v5.1 in docker through container station on my qnap nas. I'm now getting a lot of "Encountered a Too Many Requests error while logging in. Skipping reservation retrieval"

jdholtz commented 10 months ago

I'm getting that too on v5.1 Docker and v7.1 Docker. Not when running through python though...

jwin3 commented 10 months ago

I'm getting that too on v5.1 Docker and v7.1 Docker. Not when running through python though...

I'll probably run this in Python until this gets figured out. Let me know if you need me to try anything. I'm kind of dumb and don't know any coding, but I'll be a guinea pig and help any way I can.

hildebrau commented 10 months ago

Strange. I do have that in my config file at the first level.. but not in the accounts or reservation level..

Hm, I am able to check fares just fine with that configuration option (with an account and a reservation). When that option is in your config, are there are no lines in the logs saying "Checking fares for X flights"?

It didn't show any "Checking fares" lines upon the initial load up in v5.1... but after letting it run, I do see it logged these types of things:

Failed to log in to account with username REMOVED. Reason: Unknown. Status code: 500.

Requesting error during fare check. Too Many Requests 429. Skipping... Requesting error during fare check. Too Many Requests 429. Skipping... Requesting error during fare check. Too Many Requests 429. Skipping... [...repeating numerous times...]

This is in docker. I am not sure if I could run it in straight up python on my Synology.

Anyway... Have you tried Chrome 109 and undetected-chromedriver v3.1.7 yet as mentioned in that other project I linked to earlier?

jdholtz commented 10 months ago

It didn't show any "Checking fares" lines upon the initial load up in v5.1... but after letting it run, I do see it logged these types of things:

The "Checking fares" message would appear if you ran the script with the --verbose flag or looked in the file /app/logs/auto-southwest-check-in.log.

Anyway... Have you tried Chrome 109 and undetected-chromedriver v3.1.7 yet as mentioned in that other project I linked to earlier?

v5.1 does use undetected-chromedriver v3.1.7 with Chrome 115. I don't want to use undetected-chromedriver though because it is not maintained and SeleniumBase is a much better project (and is very well maintained).