App is no longer pulling updated prices from the Southwest site.

[razzamatazm]

It seems that there has been a redesign on SW side of things and the app is no longer updating prices.

From the logs - it appears SW is showing the price as infinity. I purposely increased the price above the current price and it's not giving me updates:

2018-04-24T21:04:02.235741+00:00 app[scheduler.1300]: > southwest-price-drop-bot@2.1.0 task:check /app

2018-04-24T21:04:02.235743+00:00 app[scheduler.1300]: > node --trace-warnings tasks/check.js

2018-04-24T21:04:02.235745+00:00 app[scheduler.1300]:

2018-04-24T21:04:04.134700+00:00 app[scheduler.1300]: checking 1 flights

2018-04-24T21:04:05.151598+00:00 app[scheduler.1300]: Infinity

2018-04-24T21:04:05.156428+00:00 app[scheduler.1300]: Got price: 10/26/2018|LAX|LAS|552 { time: 1524603844150, price: Infinity }

2018-04-24T21:04:05.162539+00:00 app[scheduler.1300]: 10/26/2018 #552 LAX → LAS not cheaper

There is a similar project that looks like they figured out the issue here: https://github.com/wcrasta/SWA-Scraper

[samyun]

Awesome, thanks for that link. That'll help quite a bit. I have a couple of hours at the airport tonight, so hopefully I can crank something out then.

[razzamatazm]

Thanks Sam - this app has already saved me literally thousands of miles, so it's really appreciated!

On Wed, Apr 25, 2018 at 2:36 PM Samuel Yun notifications@github.com wrote:

Awesome, thanks for that link. That'll help quite a bit. I have a couple of hours at the airport tonight, so hopefully I can crank something out then.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/samyun/southwest-price-drop-bot/issues/4#issuecomment-384442094, or mute the thread https://github.com/notifications/unsubscribe-auth/AVxCpnPSp9j-_1q2q09sJcN3wcGM_Esmks5tsOxDgaJpZM4Tjpom .

[samyun]

Unfortunately, this change isn't as simple as it seems. It seems SW did some stuff so that the POST/GET request generated doesn't return the actual page. This will take some time to investigate.

[samyun]

Instead of wasting more time figuring out Osmosis and how to work around the issue, I'm refactoring to use Puppeteer/Chrome to interact with the elements, which seems promising. If all goes well, I should have a commit up in the next few hours.

[razzamatazm]

Excellent! It looked like a pretty huge change from what I saw.

On Apr 28, 2018, at 11:14 AM, Samuel Yun notifications@github.com wrote:

Instead of wasting more time figuring out Osmosis and how to work around the issue, I'm refactoring to use Puppeteer/Chrome to interact with the elements, which seems promising. If all goes well, I should have a commit up in the next few hours.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[samyun]

https://github.com/samyun/southwest-price-drop-bot/pull/5 Pull request with the fix

[rusd1123]

I previously deployed this app with no problem, with the new commits I am getting an application error. As a note, I ran this as a new deployment and separate instance, it was not a redeploy. Looked like Puppeteer buildpack was included for new deployments. Am I missing something?

Thanks for your work on this today, @samyun !

[samyun]

You need to separately add the build pack.

Regards, Sam Yun

On Apr 28, 2018, 4:06 PM -0700, rusd1123 notifications@github.com, wrote:

@samyun I previously deployed this app with no problem, with the new commits I am getting an application error. As a note, I ran this as a new deployment and separate instance, it was not a redeploy. Looked like Puppeteer buildpack was included for new deployments. Am I missing something? Thanks for your work on this today, @samyun ! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[rusd1123]

It originally showed Google Chrome on the dashboard instead of Node.js. I deleted the Puppeteer buildpack that was already installed, Heroku then automatically added Node.js, I then reinstalled the Puppeteer buildpack and it is working. The only difference now is that It shows both Node.js and Puppeteer under the site's buildpacks.

[tjs198]

i deployed the new app and get an application error when trying to open also. I found the buildpacks under settings and the puppeteer is already there. not sure how to add node.js back in like rusd1123 mentions.

Anyone willing to help me? (not very literate with this)

[rusd1123]

@tjs198 You can select add buildpack and then select node.js and then redeploy (Node.js then Puppeteer is the order under my buildpacks). That got me past the Application Error splash page, I am still getting the following error when it is pulling the price of the flights:

Apr 28 19:22:09 swaprice app/api: Starting process with command `npm run task:check` by user scheduler@addons.heroku.com 
Apr 28 19:22:21 swaprice heroku/scheduler.6190: Starting process with command `npm run task:check` 
Apr 28 19:22:23 swaprice heroku/scheduler.6190: State changed from starting to up 
Apr 28 19:22:24 swaprice app/scheduler.6190: > southwest-price-drop-bot@3.0.0 task:check /app 
Apr 28 19:22:24 swaprice app/scheduler.6190: > node --trace-warnings tasks/check.js 
Apr 28 19:22:25 swaprice app/scheduler.6190: checking 6 flights 
Apr 28 19:22:25 swaprice app/scheduler.6190: (node:20) UnhandledPromiseRejectionWarning: Error: Failed to launch chrome! 
Apr 28 19:22:25 swaprice app/scheduler.6190: /app/node_modules/puppeteer/.local-chromium/linux-549031/chrome-linux/chrome: error while loading shared libraries: libX11-xcb.so.1: cannot open shared object file: No such file or directory 
Apr 28 19:22:25 swaprice app/scheduler.6190: TROUBLESHOOTING: https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md 
Apr 28 19:22:25 swaprice app/scheduler.6190:     at onClose (/app/node_modules/puppeteer/lib/Launcher.js:255:14) 
Apr 28 19:22:25 swaprice app/scheduler.6190:     at Interface.helper.addEventListener (/app/node_modules/puppeteer/lib/Launcher.js:244:50) 
Apr 28 19:22:25 swaprice app/scheduler.6190:     at emitNone (events.js:111:20) 
Apr 28 19:22:25 swaprice app/scheduler.6190:     at Interface.emit (events.js:208:7) 
Apr 28 19:22:25 swaprice app/scheduler.6190:     at Interface.close (readline.js:370:8) 
Apr 28 19:22:25 swaprice app/scheduler.6190:     at Socket.onend (readline.js:149:10) 
Apr 28 19:22:25 swaprice app/scheduler.6190:     at emitNone (events.js:111:20) 
Apr 28 19:22:25 swaprice app/scheduler.6190:     at Socket.emit (events.js:208:7) 
Apr 28 19:22:25 swaprice app/scheduler.6190:     at endReadableNT (_stream_readable.js:1064:12) 
Apr 28 19:22:25 swaprice app/scheduler.6190:     at _combinedTickCallback (internal/process/next_tick.js:138:11) 
Apr 28 19:22:25 swaprice app/scheduler.6190:     at process._tickCallback (internal/process/next_tick.js:180:9) 
Apr 28 19:22:25 swaprice app/scheduler.6190:     at emitWarning (internal/process/promises.js:65:17) 
Apr 28 19:22:25 swaprice app/scheduler.6190:     at emitPendingUnhandledRejections (internal/process/promises.js:109:11) 
Apr 28 19:22:25 swaprice app/scheduler.6190:     at process._tickCallback (internal/process/next_tick.js:189:7) 
Apr 28 19:22:25 swaprice app/scheduler.6190: (node:20) Error: Failed to launch chrome! 
Apr 28 19:22:25 swaprice app/scheduler.6190: /app/node_modules/puppeteer/.local-chromium/linux-549031/chrome-linux/chrome: error while loading shared libraries: libX11-xcb.so.1: cannot open shared object file: No such file or directory

If I change the order of the buildpacks to be Puppeteer then Node.js, I get the following error:

Apr 28 21:49:28 swaprice heroku/scheduler.9651:  Starting process with command `npm run task:check` 
Apr 28 21:49:28 swaprice heroku/scheduler.9651:  State changed from starting to up 
Apr 28 21:49:30 swaprice app/scheduler.9651:  > southwest-price-drop-bot@3.0.0 task:check /app 
Apr 28 21:49:30 swaprice app/scheduler.9651:  > node --trace-warnings tasks/check.js 
Apr 28 21:49:31 swaprice app/scheduler.9651:  checking 6 flights 
Apr 28 22:00:36 swaprice heroku/scheduler.9651: Cycling 
Apr 28 22:00:36 swaprice heroku/scheduler.9651: State changed from up to complete 
Apr 28 22:00:37 swaprice heroku/scheduler.9651: Stopping all processes with SIGTERM 
Apr 28 22:00:37 swaprice heroku/scheduler.9651: Process exited with status 143 

I am noticing that the task doesn't run every 10 minutes even though it is scheduled, more like every 20 minutes. I believe it is because it is taking longer than 10 minutes to "complete" the task, but not sure of the cause. @samyun any insight into the cause of these errors?

[samyun]

I'm investigating, but please open a new issue.

[rusd1123]

@tjs198 #8 should resolve your issue if you deploy a fresh build, or @samyun comment from #7 if you are redeploying.

[tjs198]

thanks so much!