search

Hydrospheredata / mist

Serverless proxy for Spark cluster
http://hydrosphere.io/mist/
Apache License 2.0
326 stars 68 forks source link

ERROR FunctionInfoProvider failed #545

Open danielhanbitlee opened 5 years ago

danielhanbitlee commented 5 years ago

Hi,

I downloaded mist and unpacked the tar.gz file. Then, I start mist. I get the following error:

daniel@daniel-K501UX:~/mist-1.1.2$ SPARK_HOME=$SPARK_HOME bin/mist-master start --debug true
19-06-19 17:07:35 [main] INFO ydrosphere.mist.master.Master$:13 Starting mist...
19-06-19 17:07:36 [main] INFO here.mist.master.MasterConfig$:360 Automatically update http.publicHost to 192.168.43.88
19-06-19 17:07:36 [main] INFO here.mist.master.MasterConfig$:360 Automatically update cluster.publicHost to 192.168.43.88
19-06-19 17:07:36 [main] INFO here.mist.master.MasterConfig$:360 Automatically update logs.publicHost to 192.168.43.88
19-06-19 17:07:36 [mist-akka.actor.default-dispatcher-2] INFO akka.event.slf4j.Slf4jLogger:89 Slf4jLogger started
19-06-19 17:07:36 [mist-akka.actor.default-dispatcher-2] INFO akka.remote.Remoting:107 Starting remoting
19-06-19 17:07:36 [mist-akka.actor.default-dispatcher-2] INFO akka.remote.Remoting:107 Remoting started; listening on addresses :[akka.tcp://mist@192.168.43.88:2551]
19-06-19 17:07:36 [main] INFO zaxxer.hikari.HikariDataSource:80 HikariPool-1 - Starting...
19-06-19 17:07:37 [main] INFO zaxxer.hikari.HikariDataSource:82 HikariPool-1 - Start completed.
19-06-19 17:07:37 [main] INFO e.internal.util.VersionPrinter:44 Flyway 4.1.1 by Boxfuse
19-06-19 17:07:37 [main] INFO nal.dbsupport.DbSupportFactory:44 Database: jdbc:h2:file:/home/daniel/Downloads/mist-1.1.2/recovery.db (H2 1.4)
19-06-19 17:07:37 [main] INFO re.internal.command.DbValidate:44 Successfully validated 2 migrations (execution time 00:00.013s)
19-06-19 17:07:37 [main] INFO etadatatable.MetaDataTableImpl:44 Creating Metadata table: "PUBLIC"."schema_version"
19-06-19 17:07:37 [main] INFO ore.internal.command.DbMigrate:44 Current version of schema "PUBLIC": << Empty Schema >>
19-06-19 17:07:37 [main] INFO ore.internal.command.DbMigrate:44 Migrating schema "PUBLIC" to version 1 - Create job details table
19-06-19 17:07:37 [main] INFO ore.internal.command.DbMigrate:44 Migrating schema "PUBLIC" to version 2 - alter table details rename endpoint
19-06-19 17:07:37 [main] INFO ore.internal.command.DbMigrate:44 Successfully applied 2 migrations to schema "PUBLIC" (execution time 00:00.120s).
19-06-19 17:07:37 [ForkJoinPool-1-worker-5] INFO here.mist.master.MasterServer$:270 DB started
19-06-19 17:07:37 [ForkJoinPool-1-worker-3] INFO here.mist.master.MasterServer$:270 LogsSystem started
19-06-19 17:07:40 [ForkJoinPool-1-worker-3] INFO here.mist.master.MasterServer$:270 FunctionInfoProvider started
19-06-19 17:07:41 [ForkJoinPool-1-worker-7] INFO here.mist.master.MasterServer$:270 Main service started
19-06-19 17:07:41 [ForkJoinPool-1-worker-7] INFO here.mist.master.MasterServer$:270 Http interface started
19-06-19 17:07:41 [main] INFO ydrosphere.mist.master.Master$:21 Mist master started
19-06-19 21:55:38 [mist-akka.actor.default-dispatcher-2] WARN akka.remote.RemoteWatcher:131 Detected unreachable: [akka.tcp://mist-info-provider@127.0.0.1:42707]
19-06-19 21:55:38 [mist-akka.actor.default-dispatcher-2] ERROR t.utils.akka.RestartSupervisor:143 Reference for FunctionInfoProvider was terminated. Restarting
19-06-19 21:56:03 [mist-akka.actor.default-dispatcher-2] ERROR t.utils.akka.RestartSupervisor:159 Starting child for FunctionInfoProvider failed
java.lang.IllegalStateException: Initialization of FunctionInfoProvider failed of timeout
    at io.hydrosphere.mist.master.jobs.ActorRefWaiter$IdentityActor$$anonfun$receive$1.applyOrElse(FunctionInfoProviderRunner.scala:60)
    at akka.actor.Actor$class.aroundReceive(Actor.scala:517)
    at io.hydrosphere.mist.master.jobs.ActorRefWaiter$IdentityActor.aroundReceive(FunctionInfoProviderRunner.scala:48)
    at akka.actor.ActorCell.receiveMessage(ActorCell.scala:527)
    at akka.actor.ActorCell.invoke(ActorCell.scala:496)
    at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
    at akka.dispatch.Mailbox.run(Mailbox.scala:224)
    at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
    at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
    at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
    at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
    at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
19-06-19 21:56:28 [mist-akka.actor.default-dispatcher-17] ERROR t.utils.akka.RestartSupervisor:159 Starting child for FunctionInfoProvider failed
java.lang.IllegalStateException: Initialization of FunctionInfoProvider failed of timeout
    at io.hydrosphere.mist.master.jobs.ActorRefWaiter$IdentityActor$$anonfun$receive$1.applyOrElse(FunctionInfoProviderRunner.scala:60)
    at akka.actor.Actor$class.aroundReceive(Actor.scala:517)
    at io.hydrosphere.mist.master.jobs.ActorRefWaiter$IdentityActor.aroundReceive(FunctionInfoProviderRunner.scala:48)
    at akka.actor.ActorCell.receiveMessage(ActorCell.scala:527)
    at akka.actor.ActorCell.invoke(ActorCell.scala:496)
    at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
    at akka.dispatch.Mailbox.run(Mailbox.scala:224)
    at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
    at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
    at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
    at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
    at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
19-06-19 21:56:53 [mist-akka.actor.default-dispatcher-14] ERROR t.utils.akka.RestartSupervisor:159 Starting child for FunctionInfoProvider failed
java.lang.IllegalStateException: Initialization of FunctionInfoProvider failed of timeout
    at io.hydrosphere.mist.master.jobs.ActorRefWaiter$IdentityActor$$anonfun$receive$1.applyOrElse(FunctionInfoProviderRunner.scala:60)
    at akka.actor.Actor$class.aroundReceive(Actor.scala:517)
    at io.hydrosphere.mist.master.jobs.ActorRefWaiter$IdentityActor.aroundReceive(FunctionInfoProviderRunner.scala:48)
    at akka.actor.ActorCell.receiveMessage(ActorCell.scala:527)
    at akka.actor.ActorCell.invoke(ActorCell.scala:496)
    at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
    at akka.dispatch.Mailbox.run(Mailbox.scala:224)
    at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
    at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
    at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
    at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
    at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
19-06-19 21:57:18 [mist-akka.actor.default-dispatcher-17] ERROR t.utils.akka.RestartSupervisor:159 Starting child for FunctionInfoProvider failed
java.lang.IllegalStateException: Initialization of FunctionInfoProvider failed of timeout
    at io.hydrosphere.mist.master.jobs.ActorRefWaiter$IdentityActor$$anonfun$receive$1.applyOrElse(FunctionInfoProviderRunner.scala:60)
    at akka.actor.Actor$class.aroundReceive(Actor.scala:517)
    at io.hydrosphere.mist.master.jobs.ActorRefWaiter$IdentityActor.aroundReceive(FunctionInfoProviderRunner.scala:48)
    at akka.actor.ActorCell.receiveMessage(ActorCell.scala:527)
    at akka.actor.ActorCell.invoke(ActorCell.scala:496)
    at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
    at akka.dispatch.Mailbox.run(Mailbox.scala:224)
    at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
    at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
    at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
    at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
    at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
19-06-19 21:57:44 [mist-akka.actor.default-dispatcher-2] ERROR t.utils.akka.RestartSupervisor:159 Starting child for FunctionInfoProvider failed
java.lang.IllegalStateException: Initialization of FunctionInfoProvider failed of timeout
    at io.hydrosphere.mist.master.jobs.ActorRefWaiter$IdentityActor$$anonfun$receive$1.applyOrElse(FunctionInfoProviderRunner.scala:60)
    at akka.actor.Actor$class.aroundReceive(Actor.scala:517)
    at io.hydrosphere.mist.master.jobs.ActorRefWaiter$IdentityActor.aroundReceive(FunctionInfoProviderRunner.scala:48)
    at akka.actor.ActorCell.receiveMessage(ActorCell.scala:527)
    at akka.actor.ActorCell.invoke(ActorCell.scala:496)
    at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
    at akka.dispatch.Mailbox.run(Mailbox.scala:224)
    at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
    at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
    at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
    at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
    at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
19-06-19 21:58:09 [mist-akka.actor.default-dispatcher-2] ERROR t.utils.akka.RestartSupervisor:159 Starting child for FunctionInfoProvider failed
java.lang.IllegalStateException: Initialization of FunctionInfoProvider failed of timeout
    at io.hydrosphere.mist.master.jobs.ActorRefWaiter$IdentityActor$$anonfun$receive$1.applyOrElse(FunctionInfoProviderRunner.scala:60)
    at akka.actor.Actor$class.aroundReceive(Actor.scala:517)
    at io.hydrosphere.mist.master.jobs.ActorRefWaiter$IdentityActor.aroundReceive(FunctionInfoProviderRunner.scala:48)
    at akka.actor.ActorCell.receiveMessage(ActorCell.scala:527)
    at akka.actor.ActorCell.invoke(ActorCell.scala:496)
    at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
    at akka.dispatch.Mailbox.run(Mailbox.scala:224)
    at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
    at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
    at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
    at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
    at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
19-06-19 21:58:34 [mist-akka.actor.default-dispatcher-2] ERROR t.utils.akka.RestartSupervisor:159 Starting child for FunctionInfoProvider failed
java.lang.IllegalStateException: Initialization of FunctionInfoProvider failed of timeout
    at io.hydrosphere.mist.master.jobs.ActorRefWaiter$IdentityActor$$anonfun$receive$1.applyOrElse(FunctionInfoProviderRunner.scala:60)
    at akka.actor.Actor$class.aroundReceive(Actor.scala:517)
    at io.hydrosphere.mist.master.jobs.ActorRefWaiter$IdentityActor.aroundReceive(FunctionInfoProviderRunner.scala:48)
    at akka.actor.ActorCell.receiveMessage(ActorCell.scala:527)
    at akka.actor.ActorCell.invoke(ActorCell.scala:496)
    at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
    at akka.dispatch.Mailbox.run(Mailbox.scala:224)
    at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
    at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
    at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
    at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
    at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
19-06-19 21:58:59 [mist-akka.actor.default-dispatcher-17] ERROR t.utils.akka.RestartSupervisor:159 Starting child for FunctionInfoProvider failed
java.lang.IllegalStateException: Initialization of FunctionInfoProvider failed of timeout
    at io.hydrosphere.mist.master.jobs.ActorRefWaiter$IdentityActor$$anonfun$receive$1.applyOrElse(FunctionInfoProviderRunner.scala:60)
    at akka.actor.Actor$class.aroundReceive(Actor.scala:517)
    at io.hydrosphere.mist.master.jobs.ActorRefWaiter$IdentityActor.aroundReceive(FunctionInfoProviderRunner.scala:48)
    at akka.actor.ActorCell.receiveMessage(ActorCell.scala:527)
    at akka.actor.ActorCell.invoke(ActorCell.scala:496)
    at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
    at akka.dispatch.Mailbox.run(Mailbox.scala:224)
    at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
    at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
    at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
    at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
    at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
19-06-19 21:59:24 [mist-akka.actor.default-dispatcher-35] ERROR t.utils.akka.RestartSupervisor:159 Starting child for FunctionInfoProvider failed, maxRetry reached
java.lang.IllegalStateException: Initialization of FunctionInfoProvider failed of timeout
    at io.hydrosphere.mist.master.jobs.ActorRefWaiter$IdentityActor$$anonfun$receive$1.applyOrElse(FunctionInfoProviderRunner.scala:60)
    at akka.actor.Actor$class.aroundReceive(Actor.scala:517)
    at io.hydrosphere.mist.master.jobs.ActorRefWaiter$IdentityActor.aroundReceive(FunctionInfoProviderRunner.scala:48)
    at akka.actor.ActorCell.receiveMessage(ActorCell.scala:527)
    at akka.actor.ActorCell.invoke(ActorCell.scala:496)
    at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
    at akka.dispatch.Mailbox.run(Mailbox.scala:224)
    at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
    at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
    at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
    at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
    at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

After this command, I can go to the user interface at http://localhost:2004/ui/jobs/overview.

Then, I enter the following from the command line:

curl --header "Content-Type: application/json" -X POST http://localhost:2004/jobs --data '{"pyPath": "/home/daniel/Documents/GitHub/apache_livy_/home/daniel/Documents/pyfiles/spark_pi.py", "name": "foo"}'

I get the following error:

The requested resource could not be found.

My environment is the following:

What am I doing wrong?

dos65 commented 5 years ago

Could you check if there is a log file from InfoProider at logs/infoprovider.log?

danielhanbitlee commented 5 years ago

Yes. There is a logs/infoprovider.log file.

danielhanbitlee commented 5 years ago

It says the following:

daniel@daniel-K501UX:~/mist-1.1.2/logs$ cat infoprovider.log
2019-06-19 17:07:38 WARN  Utils:66 - Your hostname, daniel-K501UX resolves to a loopback address: 127.0.1.1; using 192.168.43.88 instead (on interface wlp3s0)
2019-06-19 17:07:38 WARN  Utils:66 - Set SPARK_LOCAL_IP if you need to bind to another address
2019-06-19 17:07:38 WARN  NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2019-06-19 17:07:39 INFO  Slf4jLogger:89 - Slf4jLogger started
2019-06-19 17:07:39 INFO  Remoting:107 - Starting remoting
2019-06-19 17:07:39 INFO  Remoting:107 - Remoting started; listening on addresses :[akka.tcp://mist-info-provider@127.0.0.1:42707]
2019-06-19 21:55:38 WARN  RemoteWatcher:131 - Detected unreachable: [akka.tcp://mist@192.168.43.88:2551]
2019-06-19 21:55:38 INFO  FunctionInfoProvider$:86 - Remote system was terminated, shutdown app
2019-06-19 21:55:38 INFO  LocalActorRef:107 - Message [akka.remote.ReliableDeliverySupervisor$AttemptSysMsgRedelivery$] from Actor[akka://mist-info-provider/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-0#-72412422] to Actor[akka://mist-info-provider/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-0#-72412422] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:55:38 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Shutting down remote daemon.
2019-06-19 21:55:38 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Remote daemon shut down; proceeding with flushing remote transports.
2019-06-19 21:55:38 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Remoting shut down.
2019-06-19 21:55:45 WARN  Utils:66 - Your hostname, daniel-K501UX resolves to a loopback address: 127.0.1.1; using 192.168.86.142 instead (on interface wlp3s0)
2019-06-19 21:55:45 WARN  Utils:66 - Set SPARK_LOCAL_IP if you need to bind to another address
2019-06-19 21:55:46 WARN  NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2019-06-19 21:55:47 INFO  Slf4jLogger:89 - Slf4jLogger started
2019-06-19 21:55:48 INFO  Remoting:107 - Starting remoting
2019-06-19 21:55:48 INFO  Remoting:107 - Remoting started; listening on addresses :[akka.tcp://mist-info-provider@127.0.0.1:45161]
2019-06-19 21:55:58 ERROR FunctionInfoProvider$:73 - Couldn't resolve remote path akka.tcp://mist@192.168.43.88:2551/user/job-info-provider-register
akka.actor.ActorNotFound: Actor not found for: ActorSelection[Anchor(akka.tcp://mist@192.168.43.88:2551/), Path(/user/job-info-provider-register)]
    at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:70)
    at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:68)
    at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
    at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
    at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:73)
    at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.unbatchedExecute(Future.scala:78)
    at akka.dispatch.BatchingExecutor$class.execute(BatchingExecutor.scala:120)
    at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.execute(Future.scala:77)
    at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)
    at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)
    at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:600)
    at akka.actor.Scheduler$$anon$4.run(Scheduler.scala:140)
    at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:601)
    at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109)
    at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:599)
    at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:328)
    at akka.actor.LightArrayRevolverScheduler$$anon$4.executeBucket$1(LightArrayRevolverScheduler.scala:279)
    at akka.actor.LightArrayRevolverScheduler$$anon$4.nextTick(LightArrayRevolverScheduler.scala:283)
    at akka.actor.LightArrayRevolverScheduler$$anon$4.run(LightArrayRevolverScheduler.scala:235)
    at java.lang.Thread.run(Thread.java:748)
2019-06-19 21:55:58 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Shutting down remote daemon.
2019-06-19 21:55:58 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Remote daemon shut down; proceeding with flushing remote transports.
2019-06-19 21:55:58 INFO  RemoteActorRefProvider$RemoteDeadLetterActorRef:107 - Message [akka.actor.ActorIdentity] without sender to Actor[akka://mist-info-provider/deadLetters] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:55:58 INFO  LocalActorRef:107 - Message [akka.actor.Status$Failure] from Actor[akka://mist-info-provider/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-0/endpointWriter#-938748417] to Actor[akka://mist-info-provider/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-0/endpointWriter#-938748417] was not delivered. [2] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:55:58 INFO  LocalActorRef:107 - Message [akka.actor.Status$Failure] from Actor[akka://mist-info-provider/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-1#1658462574] to Actor[akka://mist-info-provider/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-1#1658462574] was not delivered. [3] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:55:58 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Remoting shut down.
2019-06-19 21:56:09 WARN  Utils:66 - Your hostname, daniel-K501UX resolves to a loopback address: 127.0.1.1; using 192.168.86.142 instead (on interface wlp3s0)
2019-06-19 21:56:09 WARN  Utils:66 - Set SPARK_LOCAL_IP if you need to bind to another address
2019-06-19 21:56:10 WARN  NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2019-06-19 21:56:10 INFO  Slf4jLogger:89 - Slf4jLogger started
2019-06-19 21:56:10 INFO  Remoting:107 - Starting remoting
2019-06-19 21:56:11 INFO  Remoting:107 - Remoting started; listening on addresses :[akka.tcp://mist-info-provider@127.0.0.1:44965]
2019-06-19 21:56:21 ERROR FunctionInfoProvider$:73 - Couldn't resolve remote path akka.tcp://mist@192.168.43.88:2551/user/job-info-provider-register
akka.actor.ActorNotFound: Actor not found for: ActorSelection[Anchor(akka.tcp://mist@192.168.43.88:2551/), Path(/user/job-info-provider-register)]
    at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:70)
    at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:68)
    at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
    at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
    at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:73)
    at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.unbatchedExecute(Future.scala:78)
    at akka.dispatch.BatchingExecutor$class.execute(BatchingExecutor.scala:120)
    at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.execute(Future.scala:77)
    at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)
    at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)
    at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:600)
    at akka.actor.Scheduler$$anon$4.run(Scheduler.scala:140)
    at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:601)
    at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109)
    at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:599)
    at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:328)
    at akka.actor.LightArrayRevolverScheduler$$anon$4.executeBucket$1(LightArrayRevolverScheduler.scala:279)
    at akka.actor.LightArrayRevolverScheduler$$anon$4.nextTick(LightArrayRevolverScheduler.scala:283)
    at akka.actor.LightArrayRevolverScheduler$$anon$4.run(LightArrayRevolverScheduler.scala:235)
    at java.lang.Thread.run(Thread.java:748)
2019-06-19 21:56:21 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Shutting down remote daemon.
2019-06-19 21:56:21 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Remote daemon shut down; proceeding with flushing remote transports.
2019-06-19 21:56:21 INFO  RemoteActorRefProvider$RemoteDeadLetterActorRef:107 - Message [akka.actor.ActorIdentity] without sender to Actor[akka://mist-info-provider/deadLetters] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:56:21 INFO  LocalActorRef:107 - Message [akka.actor.Status$Failure] from Actor[akka://mist-info-provider/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-0/endpointWriter#-1553604203] to Actor[akka://mist-info-provider/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-0/endpointWriter#-1553604203] was not delivered. [2] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:56:21 INFO  LocalActorRef:107 - Message [akka.actor.Status$Failure] from Actor[akka://mist-info-provider/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-1#250692705] to Actor[akka://mist-info-provider/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-1#250692705] was not delivered. [3] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:56:21 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Remoting shut down.
2019-06-19 21:56:35 WARN  Utils:66 - Your hostname, daniel-K501UX resolves to a loopback address: 127.0.1.1; using 192.168.86.142 instead (on interface wlp3s0)
2019-06-19 21:56:35 WARN  Utils:66 - Set SPARK_LOCAL_IP if you need to bind to another address
2019-06-19 21:56:35 WARN  NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2019-06-19 21:56:36 INFO  Slf4jLogger:89 - Slf4jLogger started
2019-06-19 21:56:36 INFO  Remoting:107 - Starting remoting
2019-06-19 21:56:37 INFO  Remoting:107 - Remoting started; listening on addresses :[akka.tcp://mist-info-provider@127.0.0.1:35181]
2019-06-19 21:56:47 ERROR FunctionInfoProvider$:73 - Couldn't resolve remote path akka.tcp://mist@192.168.43.88:2551/user/job-info-provider-register
akka.actor.ActorNotFound: Actor not found for: ActorSelection[Anchor(akka.tcp://mist@192.168.43.88:2551/), Path(/user/job-info-provider-register)]
    at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:70)
    at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:68)
    at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
    at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
    at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:73)
    at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.unbatchedExecute(Future.scala:78)
    at akka.dispatch.BatchingExecutor$class.execute(BatchingExecutor.scala:120)
    at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.execute(Future.scala:77)
    at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)
    at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)
    at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:600)
    at akka.actor.Scheduler$$anon$4.run(Scheduler.scala:140)
    at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:601)
    at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109)
    at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:599)
    at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:328)
    at akka.actor.LightArrayRevolverScheduler$$anon$4.executeBucket$1(LightArrayRevolverScheduler.scala:279)
    at akka.actor.LightArrayRevolverScheduler$$anon$4.nextTick(LightArrayRevolverScheduler.scala:283)
    at akka.actor.LightArrayRevolverScheduler$$anon$4.run(LightArrayRevolverScheduler.scala:235)
    at java.lang.Thread.run(Thread.java:748)
2019-06-19 21:56:47 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Shutting down remote daemon.
2019-06-19 21:56:47 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Remote daemon shut down; proceeding with flushing remote transports.
2019-06-19 21:56:47 INFO  RemoteActorRefProvider$RemoteDeadLetterActorRef:107 - Message [akka.actor.ActorIdentity] without sender to Actor[akka://mist-info-provider/deadLetters] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:56:47 INFO  LocalActorRef:107 - Message [akka.actor.Status$Failure] from Actor[akka://mist-info-provider/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-0/endpointWriter#-292684514] to Actor[akka://mist-info-provider/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-0/endpointWriter#-292684514] was not delivered. [2] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:56:47 INFO  LocalActorRef:107 - Message [akka.actor.Status$Failure] from Actor[akka://mist-info-provider/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-1#-926532382] to Actor[akka://mist-info-provider/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-1#-926532382] was not delivered. [3] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:56:47 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Remoting shut down.
2019-06-19 21:57:00 WARN  Utils:66 - Your hostname, daniel-K501UX resolves to a loopback address: 127.0.1.1; using 192.168.86.142 instead (on interface wlp3s0)
2019-06-19 21:57:00 WARN  Utils:66 - Set SPARK_LOCAL_IP if you need to bind to another address
2019-06-19 21:57:00 WARN  NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2019-06-19 21:57:02 INFO  Slf4jLogger:89 - Slf4jLogger started
2019-06-19 21:57:02 INFO  Remoting:107 - Starting remoting
2019-06-19 21:57:02 INFO  Remoting:107 - Remoting started; listening on addresses :[akka.tcp://mist-info-provider@127.0.0.1:39999]
2019-06-19 21:57:13 ERROR FunctionInfoProvider$:73 - Couldn't resolve remote path akka.tcp://mist@192.168.43.88:2551/user/job-info-provider-register
akka.actor.ActorNotFound: Actor not found for: ActorSelection[Anchor(akka.tcp://mist@192.168.43.88:2551/), Path(/user/job-info-provider-register)]
    at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:70)
    at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:68)
    at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
    at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
    at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:73)
    at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.unbatchedExecute(Future.scala:78)
    at akka.dispatch.BatchingExecutor$class.execute(BatchingExecutor.scala:120)
    at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.execute(Future.scala:77)
    at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)
    at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)
    at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:600)
    at akka.actor.Scheduler$$anon$4.run(Scheduler.scala:140)
    at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:601)
    at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109)
    at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:599)
    at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:328)
    at akka.actor.LightArrayRevolverScheduler$$anon$4.executeBucket$1(LightArrayRevolverScheduler.scala:279)
    at akka.actor.LightArrayRevolverScheduler$$anon$4.nextTick(LightArrayRevolverScheduler.scala:283)
    at akka.actor.LightArrayRevolverScheduler$$anon$4.run(LightArrayRevolverScheduler.scala:235)
    at java.lang.Thread.run(Thread.java:748)
2019-06-19 21:57:13 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Shutting down remote daemon.
2019-06-19 21:57:13 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Remote daemon shut down; proceeding with flushing remote transports.
2019-06-19 21:57:13 INFO  RemoteActorRefProvider$RemoteDeadLetterActorRef:107 - Message [akka.actor.ActorIdentity] without sender to Actor[akka://mist-info-provider/deadLetters] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:57:13 INFO  LocalActorRef:107 - Message [akka.actor.Status$Failure] from Actor[akka://mist-info-provider/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-0/endpointWriter#-814563640] to Actor[akka://mist-info-provider/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-0/endpointWriter#-814563640] was not delivered. [2] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:57:13 INFO  LocalActorRef:107 - Message [akka.actor.Status$Failure] from Actor[akka://mist-info-provider/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-1#-536112449] to Actor[akka://mist-info-provider/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-1#-536112449] was not delivered. [3] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:57:13 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Remoting shut down.
2019-06-19 21:57:24 WARN  Utils:66 - Your hostname, daniel-K501UX resolves to a loopback address: 127.0.1.1; using 192.168.86.142 instead (on interface wlp3s0)
2019-06-19 21:57:24 WARN  Utils:66 - Set SPARK_LOCAL_IP if you need to bind to another address
2019-06-19 21:57:25 WARN  NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2019-06-19 21:57:25 INFO  Slf4jLogger:89 - Slf4jLogger started
2019-06-19 21:57:26 INFO  Remoting:107 - Starting remoting
2019-06-19 21:57:26 INFO  Remoting:107 - Remoting started; listening on addresses :[akka.tcp://mist-info-provider@127.0.0.1:43335]
2019-06-19 21:57:36 ERROR FunctionInfoProvider$:73 - Couldn't resolve remote path akka.tcp://mist@192.168.43.88:2551/user/job-info-provider-register
akka.actor.ActorNotFound: Actor not found for: ActorSelection[Anchor(akka.tcp://mist@192.168.43.88:2551/), Path(/user/job-info-provider-register)]
    at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:70)
    at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:68)
    at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
    at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
    at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:73)
    at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.unbatchedExecute(Future.scala:78)
    at akka.dispatch.BatchingExecutor$class.execute(BatchingExecutor.scala:120)
    at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.execute(Future.scala:77)
    at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)
    at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)
    at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:600)
    at akka.actor.Scheduler$$anon$4.run(Scheduler.scala:140)
    at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:601)
    at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109)
    at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:599)
    at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:328)
    at akka.actor.LightArrayRevolverScheduler$$anon$4.executeBucket$1(LightArrayRevolverScheduler.scala:279)
    at akka.actor.LightArrayRevolverScheduler$$anon$4.nextTick(LightArrayRevolverScheduler.scala:283)
    at akka.actor.LightArrayRevolverScheduler$$anon$4.run(LightArrayRevolverScheduler.scala:235)
    at java.lang.Thread.run(Thread.java:748)
2019-06-19 21:57:36 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Shutting down remote daemon.
2019-06-19 21:57:36 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Remote daemon shut down; proceeding with flushing remote transports.
2019-06-19 21:57:36 INFO  RemoteActorRefProvider$RemoteDeadLetterActorRef:107 - Message [akka.actor.ActorIdentity] without sender to Actor[akka://mist-info-provider/deadLetters] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:57:36 INFO  LocalActorRef:107 - Message [akka.actor.Status$Failure] from Actor[akka://mist-info-provider/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-0/endpointWriter#2088901597] to Actor[akka://mist-info-provider/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-0/endpointWriter#2088901597] was not delivered. [2] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:57:36 INFO  LocalActorRef:107 - Message [akka.actor.Status$Failure] from Actor[akka://mist-info-provider/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-1#1182914469] to Actor[akka://mist-info-provider/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-1#1182914469] was not delivered. [3] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:57:36 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Remoting shut down.
2019-06-19 21:57:49 WARN  Utils:66 - Your hostname, daniel-K501UX resolves to a loopback address: 127.0.1.1; using 192.168.86.142 instead (on interface wlp3s0)
2019-06-19 21:57:49 WARN  Utils:66 - Set SPARK_LOCAL_IP if you need to bind to another address
2019-06-19 21:57:50 WARN  NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2019-06-19 21:57:51 INFO  Slf4jLogger:89 - Slf4jLogger started
2019-06-19 21:57:51 INFO  Remoting:107 - Starting remoting
2019-06-19 21:57:51 INFO  Remoting:107 - Remoting started; listening on addresses :[akka.tcp://mist-info-provider@127.0.0.1:32811]
2019-06-19 21:58:01 ERROR FunctionInfoProvider$:73 - Couldn't resolve remote path akka.tcp://mist@192.168.43.88:2551/user/job-info-provider-register
akka.actor.ActorNotFound: Actor not found for: ActorSelection[Anchor(akka.tcp://mist@192.168.43.88:2551/), Path(/user/job-info-provider-register)]
    at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:70)
    at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:68)
    at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
    at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
    at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:73)
    at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.unbatchedExecute(Future.scala:78)
    at akka.dispatch.BatchingExecutor$class.execute(BatchingExecutor.scala:120)
    at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.execute(Future.scala:77)
    at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)
    at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)
    at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:600)
    at akka.actor.Scheduler$$anon$4.run(Scheduler.scala:140)
    at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:601)
    at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109)
    at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:599)
    at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:328)
    at akka.actor.LightArrayRevolverScheduler$$anon$4.executeBucket$1(LightArrayRevolverScheduler.scala:279)
    at akka.actor.LightArrayRevolverScheduler$$anon$4.nextTick(LightArrayRevolverScheduler.scala:283)
    at akka.actor.LightArrayRevolverScheduler$$anon$4.run(LightArrayRevolverScheduler.scala:235)
    at java.lang.Thread.run(Thread.java:748)
2019-06-19 21:58:01 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Shutting down remote daemon.
2019-06-19 21:58:01 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Remote daemon shut down; proceeding with flushing remote transports.
2019-06-19 21:58:01 INFO  RemoteActorRefProvider$RemoteDeadLetterActorRef:107 - Message [akka.actor.ActorIdentity] without sender to Actor[akka://mist-info-provider/deadLetters] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:58:01 INFO  LocalActorRef:107 - Message [akka.remote.EndpointWriter$AckIdleCheckTimer$] from Actor[akka://mist-info-provider/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-0/endpointWriter#-1553962114] to Actor[akka://mist-info-provider/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-0/endpointWriter#-1553962114] was not delivered. [2] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:58:01 INFO  LocalActorRef:107 - Message [akka.actor.Status$Failure] from Actor[akka://mist-info-provider/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-0/endpointWriter#-1553962114] to Actor[akka://mist-info-provider/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-0/endpointWriter#-1553962114] was not delivered. [3] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:58:01 INFO  LocalActorRef:107 - Message [akka.actor.Status$Failure] from Actor[akka://mist-info-provider/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-1#-1425926725] to Actor[akka://mist-info-provider/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-1#-1425926725] was not delivered. [4] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:58:01 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Remoting shut down.
2019-06-19 21:58:14 WARN  Utils:66 - Your hostname, daniel-K501UX resolves to a loopback address: 127.0.1.1; using 192.168.86.142 instead (on interface wlp3s0)
2019-06-19 21:58:14 WARN  Utils:66 - Set SPARK_LOCAL_IP if you need to bind to another address
2019-06-19 21:58:15 WARN  NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2019-06-19 21:58:16 INFO  Slf4jLogger:89 - Slf4jLogger started
2019-06-19 21:58:16 INFO  Remoting:107 - Starting remoting
2019-06-19 21:58:16 INFO  Remoting:107 - Remoting started; listening on addresses :[akka.tcp://mist-info-provider@127.0.0.1:42585]
2019-06-19 21:58:26 ERROR FunctionInfoProvider$:73 - Couldn't resolve remote path akka.tcp://mist@192.168.43.88:2551/user/job-info-provider-register
akka.actor.ActorNotFound: Actor not found for: ActorSelection[Anchor(akka.tcp://mist@192.168.43.88:2551/), Path(/user/job-info-provider-register)]
    at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:70)
    at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:68)
    at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
    at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
    at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:73)
    at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.unbatchedExecute(Future.scala:78)
    at akka.dispatch.BatchingExecutor$class.execute(BatchingExecutor.scala:120)
    at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.execute(Future.scala:77)
    at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)
    at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)
    at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:600)
    at akka.actor.Scheduler$$anon$4.run(Scheduler.scala:140)
    at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:601)
    at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109)
    at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:599)
    at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:328)
    at akka.actor.LightArrayRevolverScheduler$$anon$4.executeBucket$1(LightArrayRevolverScheduler.scala:279)
    at akka.actor.LightArrayRevolverScheduler$$anon$4.nextTick(LightArrayRevolverScheduler.scala:283)
    at akka.actor.LightArrayRevolverScheduler$$anon$4.run(LightArrayRevolverScheduler.scala:235)
    at java.lang.Thread.run(Thread.java:748)
2019-06-19 21:58:26 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Shutting down remote daemon.
2019-06-19 21:58:26 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Remote daemon shut down; proceeding with flushing remote transports.
2019-06-19 21:58:26 INFO  RemoteActorRefProvider$RemoteDeadLetterActorRef:107 - Message [akka.actor.ActorIdentity] without sender to Actor[akka://mist-info-provider/deadLetters] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:58:26 INFO  LocalActorRef:107 - Message [akka.remote.EndpointWriter$AckIdleCheckTimer$] from Actor[akka://mist-info-provider/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-0/endpointWriter#-1224017059] to Actor[akka://mist-info-provider/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-0/endpointWriter#-1224017059] was not delivered. [2] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:58:27 INFO  LocalActorRef:107 - Message [akka.actor.Status$Failure] from Actor[akka://mist-info-provider/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-0/endpointWriter#-1224017059] to Actor[akka://mist-info-provider/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-0/endpointWriter#-1224017059] was not delivered. [3] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:58:27 INFO  LocalActorRef:107 - Message [akka.actor.Status$Failure] from Actor[akka://mist-info-provider/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-1#-1949687284] to Actor[akka://mist-info-provider/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-1#-1949687284] was not delivered. [4] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:58:27 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Remoting shut down.
2019-06-19 21:58:40 WARN  Utils:66 - Your hostname, daniel-K501UX resolves to a loopback address: 127.0.1.1; using 192.168.86.142 instead (on interface wlp3s0)
2019-06-19 21:58:40 WARN  Utils:66 - Set SPARK_LOCAL_IP if you need to bind to another address
2019-06-19 21:58:40 WARN  NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2019-06-19 21:58:41 INFO  Slf4jLogger:89 - Slf4jLogger started
2019-06-19 21:58:42 INFO  Remoting:107 - Starting remoting
2019-06-19 21:58:42 INFO  Remoting:107 - Remoting started; listening on addresses :[akka.tcp://mist-info-provider@127.0.0.1:46485]
2019-06-19 21:58:52 ERROR FunctionInfoProvider$:73 - Couldn't resolve remote path akka.tcp://mist@192.168.43.88:2551/user/job-info-provider-register
akka.actor.ActorNotFound: Actor not found for: ActorSelection[Anchor(akka.tcp://mist@192.168.43.88:2551/), Path(/user/job-info-provider-register)]
    at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:70)
    at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:68)
    at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
    at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
    at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:73)
    at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.unbatchedExecute(Future.scala:78)
    at akka.dispatch.BatchingExecutor$class.execute(BatchingExecutor.scala:120)
    at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.execute(Future.scala:77)
    at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)
    at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)
    at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:600)
    at akka.actor.Scheduler$$anon$4.run(Scheduler.scala:140)
    at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:601)
    at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109)
    at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:599)
    at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:328)
    at akka.actor.LightArrayRevolverScheduler$$anon$4.executeBucket$1(LightArrayRevolverScheduler.scala:279)
    at akka.actor.LightArrayRevolverScheduler$$anon$4.nextTick(LightArrayRevolverScheduler.scala:283)
    at akka.actor.LightArrayRevolverScheduler$$anon$4.run(LightArrayRevolverScheduler.scala:235)
    at java.lang.Thread.run(Thread.java:748)
2019-06-19 21:58:52 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Shutting down remote daemon.
2019-06-19 21:58:52 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Remote daemon shut down; proceeding with flushing remote transports.
2019-06-19 21:58:52 INFO  RemoteActorRefProvider$RemoteDeadLetterActorRef:107 - Message [akka.actor.ActorIdentity] without sender to Actor[akka://mist-info-provider/deadLetters] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:58:52 INFO  LocalActorRef:107 - Message [akka.actor.Status$Failure] from Actor[akka://mist-info-provider/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-0/endpointWriter#-1698100656] to Actor[akka://mist-info-provider/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-0/endpointWriter#-1698100656] was not delivered. [2] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:58:52 INFO  LocalActorRef:107 - Message [akka.actor.Status$Failure] from Actor[akka://mist-info-provider/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-1#-2025780201] to Actor[akka://mist-info-provider/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-1#-2025780201] was not delivered. [3] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:58:52 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Remoting shut down.
2019-06-19 21:59:05 WARN  Utils:66 - Your hostname, daniel-K501UX resolves to a loopback address: 127.0.1.1; using 192.168.86.142 instead (on interface wlp3s0)
2019-06-19 21:59:05 WARN  Utils:66 - Set SPARK_LOCAL_IP if you need to bind to another address
2019-06-19 21:59:05 WARN  NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2019-06-19 21:59:06 INFO  Slf4jLogger:89 - Slf4jLogger started
2019-06-19 21:59:06 INFO  Remoting:107 - Starting remoting
2019-06-19 21:59:06 INFO  Remoting:107 - Remoting started; listening on addresses :[akka.tcp://mist-info-provider@127.0.0.1:37671]
2019-06-19 21:59:16 ERROR FunctionInfoProvider$:73 - Couldn't resolve remote path akka.tcp://mist@192.168.43.88:2551/user/job-info-provider-register
akka.actor.ActorNotFound: Actor not found for: ActorSelection[Anchor(akka.tcp://mist@192.168.43.88:2551/), Path(/user/job-info-provider-register)]
    at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:70)
    at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:68)
    at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
    at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
    at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:73)
    at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.unbatchedExecute(Future.scala:78)
    at akka.dispatch.BatchingExecutor$class.execute(BatchingExecutor.scala:120)
    at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.execute(Future.scala:77)
    at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)
    at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)
    at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:600)
    at akka.actor.Scheduler$$anon$4.run(Scheduler.scala:140)
    at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:601)
    at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109)
    at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:599)
    at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:328)
    at akka.actor.LightArrayRevolverScheduler$$anon$4.executeBucket$1(LightArrayRevolverScheduler.scala:279)
    at akka.actor.LightArrayRevolverScheduler$$anon$4.nextTick(LightArrayRevolverScheduler.scala:283)
    at akka.actor.LightArrayRevolverScheduler$$anon$4.run(LightArrayRevolverScheduler.scala:235)
    at java.lang.Thread.run(Thread.java:748)
2019-06-19 21:59:16 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Shutting down remote daemon.
2019-06-19 21:59:16 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Remote daemon shut down; proceeding with flushing remote transports.
2019-06-19 21:59:16 INFO  RemoteActorRefProvider$RemoteDeadLetterActorRef:107 - Message [akka.actor.ActorIdentity] without sender to Actor[akka://mist-info-provider/deadLetters] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:59:16 INFO  LocalActorRef:107 - Message [akka.actor.Status$Failure] from Actor[akka://mist-info-provider/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-0/endpointWriter#1191614151] to Actor[akka://mist-info-provider/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-0/endpointWriter#1191614151] was not delivered. [2] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:59:16 INFO  LocalActorRef:107 - Message [akka.actor.Status$Failure] from Actor[akka://mist-info-provider/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-1#-1670265138] to Actor[akka://mist-info-provider/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2Fmist%40192.168.43.88%3A2551-1#-1670265138] was not delivered. [3] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2019-06-19 21:59:16 INFO  RemoteActorRefProvider$RemotingTerminator:107 - Remoting shut down.