samyun / southwest-price-drop-bot

Bot that watches Southwest flights for price drops.
Other
74 stars 41 forks source link

Fixes for SW erroring when trying to get flight page #65

Closed brussrus closed 2 years ago

brussrus commented 3 years ago

I modified the code a bit to handle the condition where SW errors when you go to the URL. When it errors, the page presents a button to submit, so I'm looking for that now after going to the page. I set it up in a race condition. You either will find the button and click it, or the page already has the flight information. I suppose clicking the button could still fail again, but I didn't want to get into a loop constantly checking for that.

The only thing I'm struggling with in my environment is phantom chrome processes not always cleaning up which then kills my machine (memory, etc.) . I thought I checked all try conditions and that any exceptions are supposed to close out the browser cleanly, but on occasion I get a process that just won't quit and sometimes it hangs the next task from running. Not certain if this is a puppeteer thing, a Chromium thing, or something weird on Windows. I'm not doing anything different with respect to the browser or loading that I'm aware of. Right now, I'm just trying to kill my Windows task if it runs longer than a period of time to help clean that up. I wish I didn't have to do that, because I thought this used to run pretty clean before.

I've also tweaked the code to handle entering number of passengers. To do so I changed the app.js, email-hander.js (for texts/emails to reflect the passenger count), as well as alert.js, and flight.js (for Schema). The form.pug was changed as well.

iloveitaly commented 3 years ago

@brussrus curious why most of the files are completely changed? Did the line endings or something else change? It's hard to view the exact changes that were made, but this looks like some great work.

brussrus commented 3 years ago

Not certain. I'm not overly GitHub savvy.

In browser.js I added the --no-zygote line to help with my scheduled task and zombie chrome processes sticking around. In mongo.js I added useUnifiedTopology: true and useCreateIndex: true to clean up a number of errors I was seeing during debug. app.js and email-handler.js were modified to include text for the number of passengers to be included in emails, etc. alert.js was modified again for number of passengers flight.js was modified to change the database schema to include passengers form.pug was modified to include a line to enter passengers on the web page get-prices.js was the big change to include number of passengers, and then a lot of the code to try and account for SW throwing up errors on the initial search. So now it looks to see if you find the prices or if it finds the search button on the page. If it finds the search button it clicks it and waits to "hopefully" get the price page. I've had times when it still fails on the search after clicking the button, but it has not been that bad, and often times the next time the job runs it has no problems. I felt it was at least better than it failing all the time.

And then for me the package.json was just a lot of newer versions of the node modules. I personally like to keep modules more current, even if it doesn't buy anything. However, when it comes to puppetter and its extras, I do feel the newer versions are valuable.

brussrus commented 3 years ago

The other thing that could have happened is that I uploaded some of these files manually via the web awhile back cause I have been running the number of passengers addition for over a year now and perhaps that is what accounts for some of the files looking like the whole thing was changed. I just recently started using the GitHub Desktop to try and push my changes back up.

samyun commented 2 years ago

Sorry for the very late delay. Can you look into seeing why the diff isn't working right?

iloveitaly commented 2 years ago

@brussrus quick reminder here to see if you can fix the diff issues! Would love to get this change merged.