search

Laughing-Man-Studios / Ski-Resort-Dashboard

A small dashboard to display some statistics about the local ski resorts I visit
MIT License
0 stars 0 forks source link

Epic Scrapers Failing In Production #96

Closed Rogibb111 closed 1 year ago

Rogibb111 commented 1 year ago

After moving the Epic scrapers from parsing the weather data header API call from different epic resort pages to scraping the html coming from the snow report page itself, the Epic scraper was still failing in production with the following error: 2023-01-21T12:50:25.758272142Z app[web.1]: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure 2023-01-21T12:50:25.758419371Z app[web.1]: at sun.security.ssl.Alert.createSSLException(Alert.java:131) 2023-01-21T12:50:25.758430752Z app[web.1]: at sun.security.ssl.Alert.createSSLException(Alert.java:117) 2023-01-21T12:50:25.758435090Z app[web.1]: at sun.security.ssl.TransportContext.fatal(TransportContext.java:311) 2023-01-21T12:50:25.758438937Z app[web.1]: at sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:293) 2023-01-21T12:50:25.758442574Z app[web.1]: at sun.security.ssl.TransportContext.dispatch(TransportContext.java:185) 2023-01-21T12:50:25.758446021Z app[web.1]: at sun.security.ssl.SSLTransport.decode(SSLTransport.java:152) 2023-01-21T12:50:25.758449287Z app[web.1]: at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1397) 2023-01-21T12:50:25.758452573Z app[web.1]: at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1305) 2023-01-21T12:50:25.758455749Z app[web.1]: at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:440) 2023-01-21T12:50:25.758499863Z app[web.1]: at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559) 2023-01-21T12:50:25.758510462Z app[web.1]: at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:197) 2023-01-21T12:50:25.758514099Z app[web.1]: at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:167) 2023-01-21T12:50:25.758517365Z app[web.1]: at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:732) 2023-01-21T12:50:25.758520541Z app[web.1]: at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:707) 2023-01-21T12:50:25.758523667Z app[web.1]: at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:297) 2023-01-21T12:50:25.758526853Z app[web.1]: at net.ruippeixotog.scalascraper.browser.JsoupBrowser.executeRequest(JsoupBrowser.scala:73) 2023-01-21T12:50:25.758529989Z app[web.1]: at net.ruippeixotog.scalascraper.browser.JsoupBrowser.$anonfun$executePipeline$3(JsoupBrowser.scala:84) 2023-01-21T12:50:25.758533155Z app[web.1]: at scala.Function1.$anonfun$andThen$1(Function1.scala:85) 2023-01-21T12:50:25.758552030Z app[web.1]: at scala.Function1.$anonfun$andThen$1(Function1.scala:85) 2023-01-21T12:50:25.758595854Z app[web.1]: at net.ruippeixotog.scalascraper.browser.JsoupBrowser.get(JsoupBrowser.scala:35) 2023-01-21T12:50:25.758606554Z app[web.1]: at net.ruippeixotog.scalascraper.browser.JsoupBrowser.get(JsoupBrowser.scala:29) 2023-01-21T12:50:25.758609930Z app[web.1]: at scrapers.EpicScraper.<init>(EpicScraper.scala:15) 2023-01-21T12:50:25.758613487Z app[web.1]: at scrapers.ScraperFactory$.initializeScraper(BaseScraper.scala:25) 2023-01-21T12:50:25.758731639Z app[web.1]: at controllers.HomeController.$anonfun$generateFuture$1(HomeController.scala:67) 2023-01-21T12:50:25.758741919Z app[web.1]: at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) 2023-01-21T12:50:25.758745455Z app[web.1]: at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:678) 2023-01-21T12:50:25.758748581Z app[web.1]: at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:467) 2023-01-21T12:50:25.758751668Z app[web.1]: at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402) 2023-01-21T12:50:25.758754824Z app[web.1]: at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) 2023-01-21T12:50:25.758757910Z app[web.1]: at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) 2023-01-21T12:50:25.758761086Z app[web.1]: at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) 2023-01-21T12:50:25.758764242Z app[web.1]: at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) 2023-01-21T12:50:25.758849482Z app[web.1]: () 2023-01-21T12:50:26.507597998Z app[web.1]: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure 2023-01-21T12:50:26.507750186Z app[web.1]: () 2023-01-21T12:50:26.507815147Z app[web.1]: at sun.security.ssl.Alert.createSSLException(Alert.java:131) 2023-01-21T12:50:26.507825577Z app[web.1]: at sun.security.ssl.Alert.createSSLException(Alert.java:117) 2023-01-21T12:50:26.507829626Z app[web.1]: at sun.security.ssl.TransportContext.fatal(TransportContext.java:311) 2023-01-21T12:50:26.507833222Z app[web.1]: at sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:293) 2023-01-21T12:50:26.507836528Z app[web.1]: at sun.security.ssl.TransportContext.dispatch(TransportContext.java:185) 2023-01-21T12:50:26.507839765Z app[web.1]: at sun.security.ssl.SSLTransport.decode(SSLTransport.java:152) 2023-01-21T12:50:26.507842961Z app[web.1]: at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1397) 2023-01-21T12:50:26.507846116Z app[web.1]: at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1305) 2023-01-21T12:50:26.507849272Z app[web.1]: at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:440) 2023-01-21T12:50:26.507852408Z app[web.1]: at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559) 2023-01-21T12:50:26.507855524Z app[web.1]: at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:197) 2023-01-21T12:50:26.507916840Z app[web.1]: at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:167) 2023-01-21T12:50:26.507926218Z app[web.1]: at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:732) 2023-01-21T12:50:26.507929654Z app[web.1]: at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:707) 2023-01-21T12:50:26.507932880Z app[web.1]: at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:297) 2023-01-21T12:50:26.507936046Z app[web.1]: at net.ruippeixotog.scalascraper.browser.JsoupBrowser.executeRequest(JsoupBrowser.scala:73) 2023-01-21T12:50:26.507939252Z app[web.1]: at net.ruippeixotog.scalascraper.browser.JsoupBrowser.$anonfun$executePipeline$3(JsoupBrowser.scala:84) 2023-01-21T12:50:26.507942458Z app[web.1]: at scala.Function1.$anonfun$andThen$1(Function1.scala:85) 2023-01-21T12:50:26.507945544Z app[web.1]: at scala.Function1.$anonfun$andThen$1(Function1.scala:85) 2023-01-21T12:50:26.507948700Z app[web.1]: at net.ruippeixotog.scalascraper.browser.JsoupBrowser.get(JsoupBrowser.scala:35) 2023-01-21T12:50:26.507951876Z app[web.1]: at net.ruippeixotog.scalascraper.browser.JsoupBrowser.get(JsoupBrowser.scala:29) 2023-01-21T12:50:26.507954992Z app[web.1]: at scrapers.EpicScraper.<init>(EpicScraper.scala:15) 2023-01-21T12:50:26.507958378Z app[web.1]: at scrapers.ScraperFactory$.initializeScraper(BaseScraper.scala:26) 2023-01-21T12:50:26.507961484Z app[web.1]: at controllers.HomeController.$anonfun$generateFuture$1(HomeController.scala:67) 2023-01-21T12:50:26.507964610Z app[web.1]: at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) 2023-01-21T12:50:26.507967756Z app[web.1]: at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:678) 2023-01-21T12:50:26.508010998Z app[web.1]: at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:467) 2023-01-21T12:50:26.508021788Z app[web.1]: at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402) 2023-01-21T12:50:26.508025385Z app[web.1]: at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) 2023-01-21T12:50:26.508028781Z app[web.1]: at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) 2023-01-21T12:50:26.508032007Z app[web.1]: at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) 2023-01-21T12:50:26.508035123Z app[web.1]: at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) 2023-01-21T12:50:29.535235169Z app[web.1]: 2023-01-21 12:50:29 ERROR p.api.http.DefaultHttpErrorHandler 2023-01-21T12:50:29.535301484Z app[web.1]: 2023-01-21T12:50:29.535310371Z app[web.1]: ! @80bocbdka - Internal server error, for (GET) [/scrape] -> 2023-01-21T12:50:29.535314419Z app[web.1]: 2023-01-21T12:50:29.535317945Z app[web.1]: play.api.UnexpectedException: Unexpected exception[Exception: Resort Breckenridgewasn't added to the database snapshots Map for storage. Something maybe wrong with the data scraping for said resort] 2023-01-21T12:50:29.535321722Z app[web.1]: at play.api.http.HttpErrorHandlerExceptions$.throwableToUsefulException(HttpErrorHandler.scala:358) 2023-01-21T12:50:29.535337933Z app[web.1]: at play.api.http.DefaultHttpErrorHandler.onServerError(HttpErrorHandler.scala:264) 2023-01-21T12:50:29.535341541Z app[web.1]: at play.core.server.AkkaHttpServer$$anonfun$2.applyOrElse(AkkaHttpServer.scala:430) 2023-01-21T12:50:29.535344827Z app[web.1]: at play.core.server.AkkaHttpServer$$anonfun$2.applyOrElse(AkkaHttpServer.scala:422) 2023-01-21T12:50:29.535348053Z app[web.1]: at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:490) 2023-01-21T12:50:29.535351209Z app[web.1]: at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:63) 2023-01-21T12:50:29.535401253Z app[web.1]: at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:100) 2023-01-21T12:50:29.535410490Z app[web.1]: at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) 2023-01-21T12:50:29.535414006Z app[web.1]: at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94) 2023-01-21T12:50:29.535417223Z app[web.1]: at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:100) 2023-01-21T12:50:29.535420429Z app[web.1]: at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49) 2023-01-21T12:50:29.535423675Z app[web.1]: at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48) 2023-01-21T12:50:29.535426872Z app[web.1]: at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) 2023-01-21T12:50:29.535430058Z app[web.1]: at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) 2023-01-21T12:50:29.535433203Z app[web.1]: at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) 2023-01-21T12:50:29.535436299Z app[web.1]: at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) 2023-01-21T12:50:29.535439475Z app[web.1]: Caused by: java.lang.Exception: Resort Breckenridgewasn't added to the database snapshots Map for storage. Something maybe wrong with the data scraping for said resort 2023-01-21T12:50:29.535442771Z app[web.1]: at dao.ResortData.getResortSnapshot(ResortData.scala:86) 2023-01-21T12:50:29.535445907Z app[web.1]: at dao.ResortData.setSnapshotForResort(ResortData.scala:67) 2023-01-21T12:50:29.535448973Z app[web.1]: at controllers.HomeController.$anonfun$scrape$2(HomeController.scala:59) 2023-01-21T12:50:29.535453241Z app[web.1]: at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:467) 2023-01-21T12:50:29.535503756Z app[web.1]: at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402) 2023-01-21T12:50:29.535507964Z app[web.1]: ... 4 common frames omitted

Rogibb111 commented 1 year ago

:tada: This issue has been resolved in version 1.3.4 :tada:

The release is available on GitHub release

Your semantic-release bot :package::rocket: