KryoException: Buffer overflow. Available: 1, required: 4

OryxProject / oryx

Oryx 2: Lambda architecture on Apache Spark, Apache Kafka for real-time large scale machine learning

Apache License 2.0

1.79k stars 405 forks source link

Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: acks = 1 Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: batch.size = 16384 Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: bootstrap.servers = [127.0.0.1:9092] Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: buffer.memory = 33554432 Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: client.id = Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: compression.type = gzip Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: connections.max.idle.ms = 540000 Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: enable.idempotence = false Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: interceptor.classes = null Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: key.serializer = class org.apache.kafka.common.serialization.StringSerializer Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: linger.ms = 1000 Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: max.block.ms = 60000 Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: max.in.flight.requests.per.connection = 5 Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: max.request.size = 67108864 Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: metadata.max.age.ms = 300000 Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: metric.reporters = [] Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: metrics.num.samples = 2 Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: metrics.recording.level = INFO Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: metrics.sample.window.ms = 30000 Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: receive.buffer.bytes = 32768 Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: reconnect.backoff.max.ms = 1000 Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: reconnect.backoff.ms = 50 Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: request.timeout.ms = 30000 Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: retries = 0 Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: retry.backoff.ms = 100 Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: sasl.jaas.config = null Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: sasl.kerberos.kinit.cmd = /usr/bin/kinit Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: sasl.kerberos.min.time.before.relogin = 60000 Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: sasl.kerberos.service.name = null Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: sasl.kerberos.ticket.renew.jitter = 0.05 Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: sasl.kerberos.ticket.renew.window.factor = 0.8 Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: sasl.mechanism = GSSAPI Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: security.protocol = PLAINTEXT Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: send.buffer.bytes = 131072 Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.cipher.suites = null Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1] Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.endpoint.identification.algorithm = null Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.key.password = null Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.keymanager.algorithm = SunX509 Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.keystore.location = null Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.keystore.password = null Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.keystore.type = JKS Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.protocol = TLS Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.provider = null Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.secure.random.implementation = null Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.trustmanager.algorithm = PKIX Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.truststore.location = null Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.truststore.password = null Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.truststore.type = JKS Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: transaction.timeout.ms = 60000 Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: transactional.id = null Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: value.serializer = class org.apache.kafka.common.serialization.StringSerializer Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:42:59 INFO AppInfoParser:83 - Kafka version : 0.11.0.1 Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:42:59 INFO AppInfoParser:84 - Kafka commitId : c2a0d5f9b1f45bf5 Sep 03 09:43:03 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:03 INFO ContextCleaner:54 - Cleaned shuffle 9 Sep 03 09:43:03 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:03 INFO ContextCleaner:54 - Cleaned shuffle 8 Sep 03 09:43:03 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:03 INFO KafkaProducer:1017 - Closing the Kafka producer with timeoutMillis = 9223372036854775807 ms. Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO JobScheduler:54 - Finished job streaming job 1535956800000 ms.0 from job set of time 1535956800000 ms Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO JobScheduler:54 - Starting job streaming job 1535956800000 ms.1 from job set of time 1535956800000 ms Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO UpdateOffsetsFn:60 - Updating offsets: {OryxInput,0=2477630, OryxInput,1=2478826, OryxInput,2=2477739, OryxInput,3=2478777} Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO ZooKeeper:438 - Initiating client connection, connectString=127.0.0.1:2181 sessionTimeout=30000 watcher=org.I0Itec.zkclient.ZkClient@401052bb Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO ZkEventThread:65 - Starting ZkClient event thread. Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO ZkClient:936 - Waiting for keeper state SyncConnected Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO ClientCnxn:975 - Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO ClientCnxn:852 - Socket connection established to 127.0.0.1/127.0.0.1:2181, initiating session Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO ClientCnxn:1235 - Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x10160d98a790136, negotiated timeout = 30000 Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO ZkClient:713 - zookeeper state changed (SyncConnected) Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO ZkEventThread:83 - Terminate ZkClient event thread. Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO ZooKeeper:684 - Session: 0x10160d98a790136 closed Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO ClientCnxn:512 - EventThread shut down Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO JobScheduler:54 - Finished job streaming job 1535956800000 ms.1 from job set of time 1535956800000 ms Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO JobScheduler:54 - Total delay: 184.087 s for time 1535956800000 ms (execution: 161.298 s) Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO MapPartitionsRDD:54 - Removing RDD 293 from persistence list Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO BlockManager:54 - Removing RDD 293 Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO KafkaRDD:54 - Removing RDD 292 from persistence list Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO BlockManager:54 - Removing RDD 292 Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO ReceivedBlockTracker:54 - Deleting batches: Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO InputInfoTracker:54 - remove old batch metadata: 1535956200000 ms Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO JobScheduler:54 - Added jobs for time 1535957100000 ms Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO JobScheduler:54 - Starting job streaming job 1535957100000 ms.0 from job set of time 1535957100000 ms Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO SparkContext:54 - Starting job: isEmpty at SpeedLayerUpdate.java:53 Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - Got job 264 (isEmpty at SpeedLayerUpdate.java:53) with 1 output partitions Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - Final stage: ResultStage 274 (isEmpty at SpeedLayerUpdate.java:53) Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - Parents of final stage: List() Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - Missing parents: List() Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - Submitting ResultStage 274 (MapPartitionsRDD[321] at mapToPair at SpeedLayer.java:97), which has no missing parents Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO MemoryStore:54 - Block broadcast_274 stored as values in memory (estimated size 3.8 KB, free 912.3 MB) Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO MemoryStore:54 - Block broadcast_274_piece0 stored as bytes in memory (estimated size 2.4 KB, free 912.3 MB) Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO BlockManagerInfo:54 - Added broadcast_274_piece0 in memory on htm-psycho-401.zxz.su:45401 (size: 2.4 KB, free: 912.3 MB) Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO SparkContext:54 - Created broadcast 274 from broadcast at DAGScheduler.scala:1039 Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 274 (MapPartitionsRDD[321] at mapToPair at SpeedLayer.java:97) (first 15 tasks are for partitions Vector(0)) Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO YarnScheduler:54 - Adding task set 274.0 with 1 tasks Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO TaskSetManager:54 - Starting task 0.0 in stage 274.0 (TID 586, htm-psycho-401.zxz.su, executor 1, partition 0, PROCESS_LOCAL, 7746 bytes) Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO BlockManagerInfo:54 - Added broadcast_274_piece0 in memory on htm-psycho-401.zxz.su:45269 (size: 2.4 KB, free: 912.3 MB) Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO TaskSetManager:54 - Finished task 0.0 in stage 274.0 (TID 586) in 38 ms on htm-psycho-401.zxz.su (executor 1) (1/1) Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO YarnScheduler:54 - Removed TaskSet 274.0, whose tasks have all completed, from pool Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - ResultStage 274 (isEmpty at SpeedLayerUpdate.java:53) finished in 0.049 s Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - Job 264 finished: isEmpty at SpeedLayerUpdate.java:53, took 0.052823 s Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO SparkContext:54 - Starting job: isEmpty at SpeedLayerUpdate.java:53 Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - Got job 265 (isEmpty at SpeedLayerUpdate.java:53) with 3 output partitions Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - Final stage: ResultStage 275 (isEmpty at SpeedLayerUpdate.java:53) Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - Parents of final stage: List() Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - Missing parents: List() Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - Submitting ResultStage 275 (MapPartitionsRDD[321] at mapToPair at SpeedLayer.java:97), which has no missing parents Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO MemoryStore:54 - Block broadcast_275 stored as values in memory (estimated size 3.8 KB, free 912.3 MB) Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO MemoryStore:54 - Block broadcast_275_piece0 stored as bytes in memory (estimated size 2.4 KB, free 912.3 MB) Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO BlockManagerInfo:54 - Added broadcast_275_piece0 in memory on htm-psycho-401.zxz.su:45401 (size: 2.4 KB, free: 912.3 MB) Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO SparkContext:54 - Created broadcast 275 from broadcast at DAGScheduler.scala:1039 Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - Submitting 3 missing tasks from ResultStage 275 (MapPartitionsRDD[321] at mapToPair at SpeedLayer.java:97) (first 15 tasks are for partitions Vector(1, 2, 3)) Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO YarnScheduler:54 - Adding task set 275.0 with 3 tasks Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO TaskSetManager:54 - Starting task 0.0 in stage 275.0 (TID 587, htm-psycho-401.zxz.su, executor 1, partition 1, PROCESS_LOCAL, 7746 bytes) Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO BlockManagerInfo:54 - Added broadcast_275_piece0 in memory on htm-psycho-401.zxz.su:45269 (size: 2.4 KB, free: 912.3 MB) Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO TaskSetManager:54 - Starting task 1.0 in stage 275.0 (TID 588, htm-psycho-401.zxz.su, executor 1, partition 2, PROCESS_LOCAL, 7746 bytes) Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO TaskSetManager:54 - Finished task 0.0 in stage 275.0 (TID 587) in 17 ms on htm-psycho-401.zxz.su (executor 1) (1/3) Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO TaskSetManager:54 - Starting task 2.0 in stage 275.0 (TID 589, htm-psycho-401.zxz.su, executor 1, partition 3, PROCESS_LOCAL, 7746 bytes) Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO TaskSetManager:54 - Finished task 1.0 in stage 275.0 (TID 588) in 8 ms on htm-psycho-401.zxz.su (executor 1) (2/3) Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO TaskSetManager:54 - Finished task 2.0 in stage 275.0 (TID 589) in 7 ms on htm-psycho-401.zxz.su (executor 1) (3/3) Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO YarnScheduler:54 - Removed TaskSet 275.0, whose tasks have all completed, from pool Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - ResultStage 275 (isEmpty at SpeedLayerUpdate.java:53) finished in 0.047 s Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - Job 265 finished: isEmpty at SpeedLayerUpdate.java:53, took 0.050518 s Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO JobScheduler:54 - Finished job streaming job 1535957100000 ms.0 from job set of time 1535957100000 ms Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO JobScheduler:54 - Starting job streaming job 1535957100000 ms.1 from job set of time 1535957100000 ms Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO UpdateOffsetsFn:60 - Updating offsets: {OryxInput,0=2477630, OryxInput,1=2478826, OryxInput,2=2477739, OryxInput,3=2478777} Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO ZooKeeper:438 - Initiating client connection, connectString=127.0.0.1:2181 sessionTimeout=30000 watcher=org.I0Itec.zkclient.ZkClient@382c4378 Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO ZkEventThread:65 - Starting ZkClient event thread. Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO ZkClient:936 - Waiting for keeper state SyncConnected Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO ClientCnxn:975 - Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO ClientCnxn:852 - Socket connection established to 127.0.0.1/127.0.0.1:2181, initiating session Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO ClientCnxn:1235 - Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x10160d98a790137, negotiated timeout = 30000 Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO ZkClient:713 - zookeeper state changed (SyncConnected) Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO ZkEventThread:83 - Terminate ZkClient event thread. Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO ZooKeeper:684 - Session: 0x10160d98a790137 closed Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO ClientCnxn:512 - EventThread shut down Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO JobScheduler:54 - Finished job streaming job 1535957100000 ms.1 from job set of time 1535957100000 ms Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO JobScheduler:54 - Total delay: 0.695 s for time 1535957100000 ms (execution: 0.189 s) Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO MapPartitionsRDD:54 - Removing RDD 307 from persistence list Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO KafkaRDD:54 - Removing RDD 306 from persistence list Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO BlockManager:54 - Removing RDD 307 Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO ReceivedBlockTracker:54 - Deleting batches: Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO InputInfoTracker:54 - remove old batch metadata: 1535956500000 ms Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO BlockManager:54 - Removing RDD 306 Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO RecurringTimer:54 - Stopped timer for JobGenerator after time 1535957400000 Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO JobScheduler:54 - Added jobs for time 1535957400000 ms Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO JobScheduler:54 - Starting job streaming job 1535957400000 ms.0 from job set of time 1535957400000 ms Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO SparkContext:54 - Starting job: isEmpty at SpeedLayerUpdate.java:53 Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - Got job 266 (isEmpty at SpeedLayerUpdate.java:53) with 1 output partitions Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - Final stage: ResultStage 276 (isEmpty at SpeedLayerUpdate.java:53) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - Parents of final stage: List() Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - Missing parents: List() Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - Submitting ResultStage 276 (MapPartitionsRDD[323] at mapToPair at SpeedLayer.java:97), which has no missing parents Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO MemoryStore:54 - Block broadcast_276 stored as values in memory (estimated size 3.8 KB, free 912.3 MB) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO JobGenerator:54 - Stopped generation timer Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO JobGenerator:54 - Waiting for jobs to be processed and checkpoints to be written Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 WARN JobGenerator:66 - Timed out while stopping the job generator (timeout = 300000) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO JobGenerator:54 - Waited for jobs to be processed and checkpoints to be written Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO JobGenerator:54 - Stopped JobGenerator Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO MemoryStore:54 - Block broadcast_276_piece0 stored as bytes in memory (estimated size 2.4 KB, free 912.3 MB) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO BlockManagerInfo:54 - Added broadcast_276_piece0 in memory on htm-psycho-401.zxz.su:45401 (size: 2.4 KB, free: 912.3 MB) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO SparkContext:54 - Created broadcast 276 from broadcast at DAGScheduler.scala:1039 Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 276 (MapPartitionsRDD[323] at mapToPair at SpeedLayer.java:97) (first 15 tasks are for partitions Vector(0)) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO YarnScheduler:54 - Adding task set 276.0 with 1 tasks Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO TaskSetManager:54 - Starting task 0.0 in stage 276.0 (TID 590, htm-psycho-401.zxz.su, executor 1, partition 0, PROCESS_LOCAL, 7746 bytes) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO BlockManagerInfo:54 - Added broadcast_276_piece0 in memory on htm-psycho-401.zxz.su:45269 (size: 2.4 KB, free: 912.3 MB) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO TaskSetManager:54 - Finished task 0.0 in stage 276.0 (TID 590) in 26 ms on htm-psycho-401.zxz.su (executor 1) (1/1) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO YarnScheduler:54 - Removed TaskSet 276.0, whose tasks have all completed, from pool Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - ResultStage 276 (isEmpty at SpeedLayerUpdate.java:53) finished in 0.036 s Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - Job 266 finished: isEmpty at SpeedLayerUpdate.java:53, took 0.040172 s Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO SparkContext:54 - Starting job: isEmpty at SpeedLayerUpdate.java:53 Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - Got job 267 (isEmpty at SpeedLayerUpdate.java:53) with 3 output partitions Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - Final stage: ResultStage 277 (isEmpty at SpeedLayerUpdate.java:53) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - Parents of final stage: List() Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - Missing parents: List() Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - Submitting ResultStage 277 (MapPartitionsRDD[323] at mapToPair at SpeedLayer.java:97), which has no missing parents Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO MemoryStore:54 - Block broadcast_277 stored as values in memory (estimated size 3.8 KB, free 912.3 MB) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO MemoryStore:54 - Block broadcast_277_piece0 stored as bytes in memory (estimated size 2.4 KB, free 912.3 MB) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO BlockManagerInfo:54 - Added broadcast_277_piece0 in memory on htm-psycho-401.zxz.su:45401 (size: 2.4 KB, free: 912.3 MB) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO SparkContext:54 - Created broadcast 277 from broadcast at DAGScheduler.scala:1039 Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - Submitting 3 missing tasks from ResultStage 277 (MapPartitionsRDD[323] at mapToPair at SpeedLayer.java:97) (first 15 tasks are for partitions Vector(1, 2, 3)) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO YarnScheduler:54 - Adding task set 277.0 with 3 tasks Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO TaskSetManager:54 - Starting task 0.0 in stage 277.0 (TID 591, htm-psycho-401.zxz.su, executor 1, partition 1, PROCESS_LOCAL, 7746 bytes) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO BlockManagerInfo:54 - Added broadcast_277_piece0 in memory on htm-psycho-401.zxz.su:45269 (size: 2.4 KB, free: 912.3 MB) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO TaskSetManager:54 - Starting task 1.0 in stage 277.0 (TID 592, htm-psycho-401.zxz.su, executor 1, partition 2, PROCESS_LOCAL, 7746 bytes) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO TaskSetManager:54 - Finished task 0.0 in stage 277.0 (TID 591) in 13 ms on htm-psycho-401.zxz.su (executor 1) (1/3) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO TaskSetManager:54 - Starting task 2.0 in stage 277.0 (TID 593, htm-psycho-401.zxz.su, executor 1, partition 3, PROCESS_LOCAL, 7746 bytes) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO TaskSetManager:54 - Finished task 1.0 in stage 277.0 (TID 592) in 6 ms on htm-psycho-401.zxz.su (executor 1) (2/3) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO TaskSetManager:54 - Finished task 2.0 in stage 277.0 (TID 593) in 8 ms on htm-psycho-401.zxz.su (executor 1) (3/3) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO YarnScheduler:54 - Removed TaskSet 277.0, whose tasks have all completed, from pool Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - ResultStage 277 (isEmpty at SpeedLayerUpdate.java:53) finished in 0.033 s Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - Job 267 finished: isEmpty at SpeedLayerUpdate.java:53, took 0.035632 s Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO JobScheduler:54 - Finished job streaming job 1535957400000 ms.0 from job set of time 1535957400000 ms Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO JobScheduler:54 - Starting job streaming job 1535957400000 ms.1 from job set of time 1535957400000 ms Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO UpdateOffsetsFn:60 - Updating offsets: {OryxInput,0=2477630, OryxInput,1=2478826, OryxInput,2=2477739, OryxInput,3=2478777} Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ZooKeeper:438 - Initiating client connection, connectString=127.0.0.1:2181 sessionTimeout=30000 watcher=org.I0Itec.zkclient.ZkClient@30c667e9 Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ZkEventThread:65 - Starting ZkClient event thread. Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ZkClient:936 - Waiting for keeper state SyncConnected Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ClientCnxn:975 - Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ClientCnxn:852 - Socket connection established to 127.0.0.1/127.0.0.1:2181, initiating session Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ClientCnxn:1235 - Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x10160d98a790139, negotiated timeout = 30000 Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ZkClient:713 - zookeeper state changed (SyncConnected) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ZkEventThread:83 - Terminate ZkClient event thread. Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ZooKeeper:684 - Session: 0x10160d98a790139 closed Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO JobScheduler:54 - Finished job streaming job 1535957400000 ms.1 from job set of time 1535957400000 ms Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ClientCnxn:512 - EventThread shut down Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO JobScheduler:54 - Total delay: 0.605 s for time 1535957400000 ms (execution: 0.099 s) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO JobScheduler:54 - Stopped JobScheduler Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ContextHandler:910 - Stopped o.s.j.s.ServletContextHandler@5a06904{/streaming,null,UNAVAILABLE,@Spark} Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ContextHandler:910 - Stopped o.s.j.s.ServletContextHandler@286866cb{/streaming/batch,null,UNAVAILABLE,@Spark} Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ContextHandler:910 - Stopped o.s.j.s.ServletContextHandler@14b95942{/static/streaming,null,UNAVAILABLE,@Spark} Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO StreamingContext:54 - StreamingContext stopped successfully Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO AbstractConnector:318 - Stopped Spark@18324f97{HTTP/1.1,[http/1.1]}{0.0.0.0:4041} Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO SparkUI:54 - Stopped Spark web UI at http://htm-psycho-401.zxz.su:4041 Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO YarnClientSchedulerBackend:54 - Interrupting monitor thread Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO YarnClientSchedulerBackend:54 - Shutting down all executors Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO YarnSchedulerBackend$YarnDriverEndpoint:54 - Asking each executor to shut down Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO SchedulerExtensionServices:54 - Stopping SchedulerExtensionServices Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: (serviceOption=None, Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: services=List(), Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: started=false) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO YarnClientSchedulerBackend:54 - Stopped Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO MapOutputTrackerMasterEndpoint:54 - MapOutputTrackerMasterEndpoint stopped! Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO MemoryStore:54 - MemoryStore cleared Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO BlockManager:54 - BlockManager stopped Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO BlockManagerMaster:54 - BlockManagerMaster stopped Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint:54 - OutputCommitCoordinator stopped! Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO SparkContext:54 - Successfully stopped SparkContext Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 268.0 failed 4 times, most recent failure: Lost task 0.3 in stage 268.0 (TID 567, htm-psycho-401.zxz.su, executor 1): org.apache.spark.SparkException: Kryo serialization failed: Buffer overflow. Available: 1, required: 4. To avoid this, increase spark.kryoserializer.buffer.max value. Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.serializer.KryoSerializerInstance.serialize(KryoSerializer.scala:350) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:393) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at java.lang.Thread.run(Thread.java:748) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: Caused by: com.esotericsoftware.kryo.KryoException: Buffer overflow. Available: 1, required: 4 Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.io.Output.require(Output.java:163) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.io.Output.writeInt(Output.java:254) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.io.Output.writeFloat(Output.java:496) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.serializers.UnsafeCacheFields$UnsafeFloatField.write(UnsafeCacheFields.java:80) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:628) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.write(DefaultArraySerializers.java:366) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.write(DefaultArraySerializers.java:307) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:628) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.serializer.KryoSerializerInstance.serialize(KryoSerializer.scala:347) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: ... 4 more Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: Driver stacktrace: Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1599) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1587) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1586) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1586) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:831) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:831) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at scala.Option.foreach(Option.scala:257) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:831) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1820) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1769) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1758) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:642) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.SparkContext.runJob(SparkContext.scala:2027) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.SparkContext.runJob(SparkContext.scala:2048) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.SparkContext.runJob(SparkContext.scala:2067) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.SparkContext.runJob(SparkContext.scala:2092) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.rdd.RDD$$anonfun$collect$1.apply(RDD.scala:939) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.rdd.RDD.withScope(RDD.scala:363) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.rdd.RDD.collect(RDD.scala:938) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.api.java.JavaRDDLike$class.collect(JavaRDDLike.scala:361) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.api.java.AbstractJavaRDDLike.collect(JavaRDDLike.scala:45) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.cloudera.oryx.app.speed.als.ALSSpeedModelManager.buildUpdates(ALSSpeedModelManager.java:182) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.cloudera.oryx.lambda.speed.SpeedLayerUpdate.call(SpeedLayerUpdate.java:56) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.cloudera.oryx.lambda.speed.SpeedLayerUpdate.call(SpeedLayerUpdate.java:37) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.api.java.JavaDStreamLike$$anonfun$foreachRDD$1.apply(JavaDStreamLike.scala:272) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.api.java.JavaDStreamLike$$anonfun$foreachRDD$1.apply(JavaDStreamLike.scala:272) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.dstream.DStream$$anonfun$foreachRDD$1$$anonfun$apply$mcV$sp$3.apply(DStream.scala:628) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.dstream.DStream$$anonfun$foreachRDD$1$$anonfun$apply$mcV$sp$3.apply(DStream.scala:628) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:416) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at scala.util.Try$.apply(Try.scala:192) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:257) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:257) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:257) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:256) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at java.lang.Thread.run(Thread.java:748) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: Caused by: org.apache.spark.SparkException: Kryo serialization failed: Buffer overflow. Available: 1, required: 4. To avoid this, increase spark.kryoserializer.buffer.max value. Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.serializer.KryoSerializerInstance.serialize(KryoSerializer.scala:350) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:393) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: ... 3 more Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: Caused by: com.esotericsoftware.kryo.KryoException: Buffer overflow. Available: 1, required: 4 Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.io.Output.require(Output.java:163) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.io.Output.writeInt(Output.java:254) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.io.Output.writeFloat(Output.java:496) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.serializers.UnsafeCacheFields$UnsafeFloatField.write(UnsafeCacheFields.java:80) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:628) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.write(DefaultArraySerializers.java:366) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.write(DefaultArraySerializers.java:307) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:628) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.serializer.KryoSerializerInstance.serialize(KryoSerializer.scala:347) Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: ... 4 more Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ShutdownHookManager:54 - Shutdown hook called Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ShutdownHookManager:54 - Deleting directory /tmp/spark-4d5c56ac-ccfc-4c15-ba08-6411e35a1e5e Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ShutdownHookManager:54 - Deleting directory /tmp/spark-fab670a2-b75d-4494-a3c1-d49046e1c765 Sep 03 09:50:01 htm-psycho-401.zxz.su systemd[1]: oryx-speed.service: main process exited, code=exited, status=1/FAILURE Sep 03 09:50:01 htm-psycho-401.zxz.su systemd[1]: Unit oryx-speed.service entered failed state. Sep 03 09:50:01 htm-psycho-401.zxz.su systemd[1]: oryx-speed.service failed.

Sep 06 11:45:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:45:00 INFO BlockManager:54 - Removing RDD 9 Sep 06 11:45:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:45:00 INFO KafkaRDD:54 - Removing RDD 8 from persistence list Sep 06 11:45:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:45:00 INFO BlockManager:54 - Removing RDD 8 Sep 06 11:45:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:45:00 INFO ReceivedBlockTracker:54 - Deleting batches: Sep 06 11:45:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:45:00 INFO InputInfoTracker:54 - remove old batch metadata: 1536222900000 ms Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:45:38 INFO YarnSchedulerBackend$YarnDriverEndpoint:54 - Disabling executor 1. Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:45:38 INFO DAGScheduler:54 - Executor lost: 1 (epoch 0) Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:45:38 INFO BlockManagerMasterEndpoint:54 - Trying to remove executor 1 from BlockManagerMaster. Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:45:38 ERROR TransportClient:233 - Failed to send RPC 7240739849994500636 to /192.168.147.4:53566: java.nio.channels.ClosedChannelException Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: java.nio.channels.ClosedChannelException Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.AbstractChannel$AbstractUnsafe.write(...)(Unknown Source) Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:45:38 INFO BlockManagerMasterEndpoint:54 - Removing block manager BlockManagerId(1, htm-psycho-401.zxz.su, 36859, None) Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:45:38 INFO BlockManagerMaster:54 - Removed 1 successfully in removeExecutor Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:45:38 WARN YarnSchedulerBackend$YarnSchedulerEndpoint:87 - Attempted to get executor loss reason for executor id 1 at RPC address 192.168.147.4:53574, but got no response. Marking as slave lost. Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: java.io.IOException: Failed to send RPC 7240739849994500636 to /192.168.147.4:53566: java.nio.channels.ClosedChannelException Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.network.client.TransportClient.lambda$sendRpc$2(TransportClient.java:237) Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507) Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:481) Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420) Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:122) Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.AbstractChannel$AbstractUnsafe.safeSetFailure(AbstractChannel.java:987) Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.AbstractChannel$AbstractUnsafe.write(AbstractChannel.java:869) Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.DefaultChannelPipeline$HeadContext.write(DefaultChannelPipeline.java:1316) Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:738) Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:730) Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.AbstractChannelHandlerContext.access$1900(AbstractChannelHandlerContext.java:38) Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.write(AbstractChannelHandlerContext.java:1081) Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.AbstractChannelHandlerContext$WriteAndFlushTask.write(AbstractChannelHandlerContext.java:1128) Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.run(AbstractChannelHandlerContext.java:1070) Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403) Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463) Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138) Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: at java.lang.Thread.run(Thread.java:748) Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: Caused by: java.nio.channels.ClosedChannelException Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.AbstractChannel$AbstractUnsafe.write(...)(Unknown Source) Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:45:38 INFO DAGScheduler:54 - Shuffle files lost for executor: 1 (epoch 0) Sep 06 11:45:38 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:45:38 ERROR YarnScheduler:70 - Lost executor 1 on htm-psycho-401.zxz.su: Slave lost Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:46:13 ERROR Client:70 - Application application_1536006464455_0008 not found. Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:46:13 INFO Client:54 - Deleted staging directory hdfs://0.0.0.0:9000/user/ec2-user/.sparkStaging/application_1536006464455_0008 Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:46:13 ERROR YarnClientSchedulerBackend:70 - Yarn application has already exited with state KILLED! Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:46:13 INFO AbstractConnector:318 - Stopped Spark@18324f97{HTTP/1.1,[http/1.1]}{0.0.0.0:4041} Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:46:13 INFO SparkUI:54 - Stopped Spark web UI at http://htm-psycho-401.zxz.su:4041 Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:46:13 ERROR TransportClient:233 - Failed to send RPC 6480775028423913532 to /192.168.147.4:53566: java.nio.channels.ClosedChannelException Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: java.nio.channels.ClosedChannelException Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.AbstractChannel$AbstractUnsafe.write(...)(Unknown Source) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:46:13 ERROR YarnSchedulerBackend$YarnSchedulerEndpoint:91 - Sending RequestExecutors(0,0,Map(),Set()) to AM was unsuccessful Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: java.io.IOException: Failed to send RPC 6480775028423913532 to /192.168.147.4:53566: java.nio.channels.ClosedChannelException Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.network.client.TransportClient.lambda$sendRpc$2(TransportClient.java:237) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:481) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:122) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.AbstractChannel$AbstractUnsafe.safeSetFailure(AbstractChannel.java:987) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.AbstractChannel$AbstractUnsafe.write(AbstractChannel.java:869) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.DefaultChannelPipeline$HeadContext.write(DefaultChannelPipeline.java:1316) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:738) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:730) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.AbstractChannelHandlerContext.access$1900(AbstractChannelHandlerContext.java:38) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.write(AbstractChannelHandlerContext.java:1081) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.AbstractChannelHandlerContext$WriteAndFlushTask.write(AbstractChannelHandlerContext.java:1128) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.run(AbstractChannelHandlerContext.java:1070) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at java.lang.Thread.run(Thread.java:748) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: Caused by: java.nio.channels.ClosedChannelException Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.AbstractChannel$AbstractUnsafe.write(...)(Unknown Source) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:46:13 INFO SchedulerExtensionServices:54 - Stopping SchedulerExtensionServices Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: (serviceOption=None, Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: services=List(), Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: started=false) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:46:13 ERROR Utils:91 - Uncaught exception in thread Yarn application state monitor Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: org.apache.spark.SparkException: Exception thrown in awaitResult: Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.util.ThreadUtils$.awaitResult(ThreadUtils.scala:205) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rpc.RpcTimeout.awaitResult(RpcTimeout.scala:75) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.scheduler.cluster.CoarseGrainedSchedulerBackend.requestTotalExecutors(CoarseGrainedSchedulerBackend.scala:566) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.scheduler.cluster.YarnSchedulerBackend.stop(YarnSchedulerBackend.scala:95) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.stop(YarnClientSchedulerBackend.scala:155) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.scheduler.TaskSchedulerImpl.stop(TaskSchedulerImpl.scala:508) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.scheduler.DAGScheduler.stop(DAGScheduler.scala:1752) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.SparkContext$$anonfun$stop$8.apply$mcV$sp(SparkContext.scala:1924) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.util.Utils$.tryLogNonFatalError(Utils.scala:1357) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.SparkContext.stop(SparkContext.scala:1923) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend$MonitorThread.run(YarnClientSchedulerBackend.scala:112) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: Caused by: java.io.IOException: Failed to send RPC 6480775028423913532 to /192.168.147.4:53566: java.nio.channels.ClosedChannelException Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.network.client.TransportClient.lambda$sendRpc$2(TransportClient.java:237) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:481) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:122) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.AbstractChannel$AbstractUnsafe.safeSetFailure(AbstractChannel.java:987) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.AbstractChannel$AbstractUnsafe.write(AbstractChannel.java:869) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.DefaultChannelPipeline$HeadContext.write(DefaultChannelPipeline.java:1316) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:738) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:730) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.AbstractChannelHandlerContext.access$1900(AbstractChannelHandlerContext.java:38) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.write(AbstractChannelHandlerContext.java:1081) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.AbstractChannelHandlerContext$WriteAndFlushTask.write(AbstractChannelHandlerContext.java:1128) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.run(AbstractChannelHandlerContext.java:1070) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at java.lang.Thread.run(Thread.java:748) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: Caused by: java.nio.channels.ClosedChannelException Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: at io.netty.channel.AbstractChannel$AbstractUnsafe.write(...)(Unknown Source) Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:46:13 INFO MapOutputTrackerMasterEndpoint:54 - MapOutputTrackerMasterEndpoint stopped! Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:46:13 INFO MemoryStore:54 - MemoryStore cleared Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:46:13 INFO BlockManager:54 - BlockManager stopped Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:46:13 INFO BlockManagerMaster:54 - BlockManagerMaster stopped Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:46:13 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint:54 - OutputCommitCoordinator stopped! Sep 06 11:46:13 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:46:13 INFO SparkContext:54 - Successfully stopped SparkContext Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:50:00 INFO JobScheduler:54 - Added jobs for time 1536223800000 ms Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:50:00 INFO JobScheduler:54 - Starting job streaming job 1536223800000 ms.0 from job set of time 1536223800000 ms Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:50:00 INFO JobScheduler:54 - Finished job streaming job 1536223800000 ms.0 from job set of time 1536223800000 ms Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:50:00 INFO UpdateOffsetsFn:60 - Updating offsets: {OryxInput,0=2477636, OryxInput,1=2478832, OryxInput,2=2477745, OryxInput,3=2478777} Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:50:00 INFO ZooKeeper:438 - Initiating client connection, connectString=127.0.0.1:2181 sessionTimeout=30000 watcher=org.I0Itec.zkclient.ZkClient@589f343b Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:50:00 INFO ZkEventThread:65 - Starting ZkClient event thread. Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:50:00 INFO JobScheduler:54 - Starting job streaming job 1536223800000 ms.1 from job set of time 1536223800000 ms Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:50:00 INFO ZkClient:936 - Waiting for keeper state SyncConnected Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:50:00 INFO ClientCnxn:975 - Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:50:00 ERROR JobScheduler:91 - Error running job streaming job 1536223800000 ms.0 Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: java.lang.IllegalStateException: SparkContext has been shutdown Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.SparkContext.runJob(SparkContext.scala:2019) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.SparkContext.runJob(SparkContext.scala:2048) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.SparkContext.runJob(SparkContext.scala:2067) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD$$anonfun$take$1.apply(RDD.scala:1358) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD.withScope(RDD.scala:363) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD.take(RDD.scala:1331) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD$$anonfun$isEmpty$1.apply$mcZ$sp(RDD.scala:1466) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD$$anonfun$isEmpty$1.apply(RDD.scala:1466) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD$$anonfun$isEmpty$1.apply(RDD.scala:1466) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD.withScope(RDD.scala:363) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD.isEmpty(RDD.scala:1465) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.api.java.JavaRDDLike$class.isEmpty(JavaRDDLike.scala:544) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.api.java.AbstractJavaRDDLike.isEmpty(JavaRDDLike.scala:45) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at com.cloudera.oryx.lambda.speed.SpeedLayerUpdate.call(SpeedLayerUpdate.java:53) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at com.cloudera.oryx.lambda.speed.SpeedLayerUpdate.call(SpeedLayerUpdate.java:37) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.api.java.JavaDStreamLike$$anonfun$foreachRDD$1.apply(JavaDStreamLike.scala:272) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.api.java.JavaDStreamLike$$anonfun$foreachRDD$1.apply(JavaDStreamLike.scala:272) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.DStream$$anonfun$foreachRDD$1$$anonfun$apply$mcV$sp$3.apply(DStream.scala:628) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.DStream$$anonfun$foreachRDD$1$$anonfun$apply$mcV$sp$3.apply(DStream.scala:628) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:416) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at scala.util.Try$.apply(Try.scala:192) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:257) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:257) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:257) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:256) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at java.lang.Thread.run(Thread.java:748) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:50:00 INFO ClientCnxn:852 - Socket connection established to 127.0.0.1/127.0.0.1:2181, initiating session Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:50:00 INFO SpeedLayer:142 - Shutting down model manager Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:50:00 INFO SpeedLayer:147 - Shutting down consumer Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:50:00 INFO SpeedLayer:152 - Shutting down Spark Streaming; this may take some time Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:50:00 INFO ClientCnxn:1235 - Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x10160d98a790153, negotiated timeout = 30000 Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:50:00 INFO ReceiverTracker:54 - ReceiverTracker stopped Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:50:00 INFO ZkClient:713 - zookeeper state changed (SyncConnected) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:50:00 INFO JobGenerator:54 - Stopping JobGenerator gracefully Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:50:00 INFO JobGenerator:54 - Waiting for all received blocks to be consumed for job generation Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:50:00 INFO JobGenerator:54 - Waited for all received blocks to be consumed for job generation Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:50:00 INFO ZkEventThread:83 - Terminate ZkClient event thread. Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:50:00 INFO ZooKeeper:684 - Session: 0x10160d98a790153 closed Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:50:00 INFO ClientCnxn:512 - EventThread shut down Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:50:00 INFO JobScheduler:54 - Finished job streaming job 1536223800000 ms.1 from job set of time 1536223800000 ms Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:50:00 INFO JobScheduler:54 - Total delay: 0.026 s for time 1536223800000 ms (execution: 0.021 s) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:50:00 INFO MapPartitionsRDD:54 - Removing RDD 11 from persistence list Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:50:00 ERROR JobScheduler:91 - Error in job generator Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: java.lang.NullPointerException Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.storage.BlockManagerMaster.removeRdd(BlockManagerMaster.scala:126) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.SparkContext.unpersistRDD(SparkContext.scala:1790) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD.unpersist(RDD.scala:217) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.DStream$$anonfun$clearMetadata$3.apply(DStream.scala:458) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.DStream$$anonfun$clearMetadata$3.apply(DStream.scala:457) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at scala.collection.mutable.HashMap$$anon$2$$anonfun$foreach$3.apply(HashMap.scala:108) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at scala.collection.mutable.HashMap$$anon$2$$anonfun$foreach$3.apply(HashMap.scala:108) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at scala.collection.mutable.HashMap$$anon$2.foreach(HashMap.scala:108) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.DStream.clearMetadata(DStream.scala:457) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.DStream$$anonfun$clearMetadata$5.apply(DStream.scala:470) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.DStream$$anonfun$clearMetadata$5.apply(DStream.scala:470) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at scala.collection.immutable.List.foreach(List.scala:381) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.DStream.clearMetadata(DStream.scala:470) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.DStreamGraph$$anonfun$clearMetadata$2.apply(DStreamGraph.scala:134) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.DStreamGraph$$anonfun$clearMetadata$2.apply(DStreamGraph.scala:134) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.DStreamGraph.clearMetadata(DStreamGraph.scala:134) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.scheduler.JobGenerator.clearMetadata(JobGenerator.scala:263) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.scheduler.JobGenerator.org$apache$spark$streaming$scheduler$JobGenerator$$processEvent(JobGenerator.scala:184) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:89) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:88) Sep 06 11:50:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:55:00 INFO JobScheduler:54 - Added jobs for time 1536224100000 ms Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:55:00 INFO JobScheduler:54 - Starting job streaming job 1536224100000 ms.0 from job set of time 1536224100000 ms Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:55:00 INFO JobScheduler:54 - Finished job streaming job 1536224100000 ms.0 from job set of time 1536224100000 ms Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:55:00 INFO UpdateOffsetsFn:60 - Updating offsets: {OryxInput,0=2477636, OryxInput,1=2478832, OryxInput,2=2477745, OryxInput,3=2478777} Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:55:00 INFO JobScheduler:54 - Starting job streaming job 1536224100000 ms.1 from job set of time 1536224100000 ms Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:55:00 ERROR JobScheduler:91 - Error running job streaming job 1536224100000 ms.0 Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: java.lang.IllegalStateException: SparkContext has been shutdown Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.SparkContext.runJob(SparkContext.scala:2019) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.SparkContext.runJob(SparkContext.scala:2048) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.SparkContext.runJob(SparkContext.scala:2067) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD$$anonfun$take$1.apply(RDD.scala:1358) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD.withScope(RDD.scala:363) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD.take(RDD.scala:1331) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD$$anonfun$isEmpty$1.apply$mcZ$sp(RDD.scala:1466) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD$$anonfun$isEmpty$1.apply(RDD.scala:1466) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD$$anonfun$isEmpty$1.apply(RDD.scala:1466) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD.withScope(RDD.scala:363) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD.isEmpty(RDD.scala:1465) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.api.java.JavaRDDLike$class.isEmpty(JavaRDDLike.scala:544) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.api.java.AbstractJavaRDDLike.isEmpty(JavaRDDLike.scala:45) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at com.cloudera.oryx.lambda.speed.SpeedLayerUpdate.call(SpeedLayerUpdate.java:53) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at com.cloudera.oryx.lambda.speed.SpeedLayerUpdate.call(SpeedLayerUpdate.java:37) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.api.java.JavaDStreamLike$$anonfun$foreachRDD$1.apply(JavaDStreamLike.scala:272) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.api.java.JavaDStreamLike$$anonfun$foreachRDD$1.apply(JavaDStreamLike.scala:272) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.DStream$$anonfun$foreachRDD$1$$anonfun$apply$mcV$sp$3.apply(DStream.scala:628) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.DStream$$anonfun$foreachRDD$1$$anonfun$apply$mcV$sp$3.apply(DStream.scala:628) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:416) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at scala.util.Try$.apply(Try.scala:192) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:257) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:257) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:257) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:256) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at java.lang.Thread.run(Thread.java:748) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:55:00 INFO ZkEventThread:65 - Starting ZkClient event thread. Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:55:00 INFO ZooKeeper:438 - Initiating client connection, connectString=127.0.0.1:2181 sessionTimeout=30000 watcher=org.I0Itec.zkclient.ZkClient@496005e2 Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:55:00 INFO ZkClient:936 - Waiting for keeper state SyncConnected Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:55:00 INFO ClientCnxn:975 - Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:55:00 INFO ClientCnxn:852 - Socket connection established to 127.0.0.1/127.0.0.1:2181, initiating session Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:55:00 INFO ClientCnxn:1235 - Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x10160d98a790156, negotiated timeout = 30000 Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:55:00 INFO ZkClient:713 - zookeeper state changed (SyncConnected) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:55:00 INFO ZkEventThread:83 - Terminate ZkClient event thread. Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:55:00 INFO ZooKeeper:684 - Session: 0x10160d98a790156 closed Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:55:00 INFO ClientCnxn:512 - EventThread shut down Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:55:00 INFO JobScheduler:54 - Finished job streaming job 1536224100000 ms.1 from job set of time 1536224100000 ms Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:55:00 INFO JobScheduler:54 - Total delay: 0.052 s for time 1536224100000 ms (execution: 0.044 s) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:55:00 INFO MapPartitionsRDD:54 - Removing RDD 13 from persistence list Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 11:55:00 ERROR JobScheduler:91 - Error in job generator Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: java.lang.NullPointerException Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.storage.BlockManagerMaster.removeRdd(BlockManagerMaster.scala:126) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.SparkContext.unpersistRDD(SparkContext.scala:1790) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD.unpersist(RDD.scala:217) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.DStream$$anonfun$clearMetadata$3.apply(DStream.scala:458) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.DStream$$anonfun$clearMetadata$3.apply(DStream.scala:457) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at scala.collection.mutable.HashMap$$anon$2$$anonfun$foreach$3.apply(HashMap.scala:108) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at scala.collection.mutable.HashMap$$anon$2$$anonfun$foreach$3.apply(HashMap.scala:108) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at scala.collection.mutable.HashMap$$anon$2.foreach(HashMap.scala:108) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.DStream.clearMetadata(DStream.scala:457) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.DStream$$anonfun$clearMetadata$5.apply(DStream.scala:470) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.DStream$$anonfun$clearMetadata$5.apply(DStream.scala:470) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at scala.collection.immutable.List.foreach(List.scala:381) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.DStream.clearMetadata(DStream.scala:470) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.DStreamGraph$$anonfun$clearMetadata$2.apply(DStreamGraph.scala:134) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.DStreamGraph$$anonfun$clearMetadata$2.apply(DStreamGraph.scala:134) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.DStreamGraph.clearMetadata(DStreamGraph.scala:134) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.scheduler.JobGenerator.clearMetadata(JobGenerator.scala:263) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.scheduler.JobGenerator.org$apache$spark$streaming$scheduler$JobGenerator$$processEvent(JobGenerator.scala:184) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:89) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:88) Sep 06 11:55:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO RecurringTimer:54 - Stopped timer for JobGenerator after time 1536224400000 Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO JobScheduler:54 - Added jobs for time 1536224400000 ms Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO JobScheduler:54 - Starting job streaming job 1536224400000 ms.0 from job set of time 1536224400000 ms Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO JobGenerator:54 - Stopped generation timer Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO JobScheduler:54 - Finished job streaming job 1536224400000 ms.0 from job set of time 1536224400000 ms Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO UpdateOffsetsFn:60 - Updating offsets: {OryxInput,0=2477636, OryxInput,1=2478832, OryxInput,2=2477745, OryxInput,3=2478777} Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO JobGenerator:54 - Waiting for jobs to be processed and checkpoints to be written Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO JobScheduler:54 - Starting job streaming job 1536224400000 ms.1 from job set of time 1536224400000 ms Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 ERROR JobScheduler:91 - Error running job streaming job 1536224400000 ms.0 Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: java.lang.IllegalStateException: SparkContext has been shutdown Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.SparkContext.runJob(SparkContext.scala:2019) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.SparkContext.runJob(SparkContext.scala:2048) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.SparkContext.runJob(SparkContext.scala:2067) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD$$anonfun$take$1.apply(RDD.scala:1358) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD.withScope(RDD.scala:363) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD.take(RDD.scala:1331) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD$$anonfun$isEmpty$1.apply$mcZ$sp(RDD.scala:1466) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD$$anonfun$isEmpty$1.apply(RDD.scala:1466) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD$$anonfun$isEmpty$1.apply(RDD.scala:1466) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD.withScope(RDD.scala:363) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD.isEmpty(RDD.scala:1465) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.api.java.JavaRDDLike$class.isEmpty(JavaRDDLike.scala:544) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.api.java.AbstractJavaRDDLike.isEmpty(JavaRDDLike.scala:45) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at com.cloudera.oryx.lambda.speed.SpeedLayerUpdate.call(SpeedLayerUpdate.java:53) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at com.cloudera.oryx.lambda.speed.SpeedLayerUpdate.call(SpeedLayerUpdate.java:37) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.api.java.JavaDStreamLike$$anonfun$foreachRDD$1.apply(JavaDStreamLike.scala:272) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.api.java.JavaDStreamLike$$anonfun$foreachRDD$1.apply(JavaDStreamLike.scala:272) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.DStream$$anonfun$foreachRDD$1$$anonfun$apply$mcV$sp$3.apply(DStream.scala:628) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.DStream$$anonfun$foreachRDD$1$$anonfun$apply$mcV$sp$3.apply(DStream.scala:628) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:416) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at scala.util.Try$.apply(Try.scala:192) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:257) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:257) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:257) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:256) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at java.lang.Thread.run(Thread.java:748) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO ZkEventThread:65 - Starting ZkClient event thread. Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 WARN JobGenerator:66 - Timed out while stopping the job generator (timeout = 300000) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO ZooKeeper:438 - Initiating client connection, connectString=127.0.0.1:2181 sessionTimeout=30000 watcher=org.I0Itec.zkclient.ZkClient@6e82d48f Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO JobGenerator:54 - Waited for jobs to be processed and checkpoints to be written Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO ZkClient:936 - Waiting for keeper state SyncConnected Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO JobGenerator:54 - Stopped JobGenerator Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO ClientCnxn:975 - Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO ClientCnxn:852 - Socket connection established to 127.0.0.1/127.0.0.1:2181, initiating session Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO ClientCnxn:1235 - Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x10160d98a79015b, negotiated timeout = 30000 Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO ZkClient:713 - zookeeper state changed (SyncConnected) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO ZkEventThread:83 - Terminate ZkClient event thread. Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO ClientCnxn:512 - EventThread shut down Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO ZooKeeper:684 - Session: 0x10160d98a79015b closed Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO JobScheduler:54 - Finished job streaming job 1536224400000 ms.1 from job set of time 1536224400000 ms Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO JobScheduler:54 - Total delay: 0.022 s for time 1536224400000 ms (execution: 0.017 s) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO JobScheduler:54 - Stopped JobScheduler Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO ContextHandler:910 - Stopped o.s.j.s.ServletContextHandler@5a06904{/streaming,null,UNAVAILABLE,@Spark} Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO ContextHandler:910 - Stopped o.s.j.s.ServletContextHandler@286866cb{/streaming/batch,null,UNAVAILABLE,@Spark} Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO ContextHandler:910 - Stopped o.s.j.s.ServletContextHandler@14b95942{/static/streaming,null,UNAVAILABLE,@Spark} Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO StreamingContext:54 - StreamingContext stopped successfully Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO SparkContext:54 - SparkContext already stopped. Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: Exception in thread "main" java.lang.IllegalStateException: SparkContext has been shutdown Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.SparkContext.runJob(SparkContext.scala:2019) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.SparkContext.runJob(SparkContext.scala:2048) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.SparkContext.runJob(SparkContext.scala:2067) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD$$anonfun$take$1.apply(RDD.scala:1358) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD.withScope(RDD.scala:363) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD.take(RDD.scala:1331) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD$$anonfun$isEmpty$1.apply$mcZ$sp(RDD.scala:1466) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD$$anonfun$isEmpty$1.apply(RDD.scala:1466) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD$$anonfun$isEmpty$1.apply(RDD.scala:1466) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD.withScope(RDD.scala:363) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.rdd.RDD.isEmpty(RDD.scala:1465) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.api.java.JavaRDDLike$class.isEmpty(JavaRDDLike.scala:544) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.api.java.AbstractJavaRDDLike.isEmpty(JavaRDDLike.scala:45) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at com.cloudera.oryx.lambda.speed.SpeedLayerUpdate.call(SpeedLayerUpdate.java:53) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at com.cloudera.oryx.lambda.speed.SpeedLayerUpdate.call(SpeedLayerUpdate.java:37) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.api.java.JavaDStreamLike$$anonfun$foreachRDD$1.apply(JavaDStreamLike.scala:272) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.api.java.JavaDStreamLike$$anonfun$foreachRDD$1.apply(JavaDStreamLike.scala:272) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.DStream$$anonfun$foreachRDD$1$$anonfun$apply$mcV$sp$3.apply(DStream.scala:628) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.DStream$$anonfun$foreachRDD$1$$anonfun$apply$mcV$sp$3.apply(DStream.scala:628) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:416) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at scala.util.Try$.apply(Try.scala:192) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:257) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:257) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:257) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:256) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: at java.lang.Thread.run(Thread.java:748) Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO ShutdownHookManager:54 - Shutdown hook called Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO ShutdownHookManager:54 - Deleting directory /tmp/spark-16850b34-eb08-469f-915f-1e97f2942c29 Sep 06 12:00:00 htm-psycho-401.zxz.su bash[29328]: 2018-09-06 12:00:00 INFO ShutdownHookManager:54 - Deleting directory /tmp/spark-e33ba689-595b-4e3e-be27-8df08053b020 Sep 06 12:00:00 htm-psycho-401.zxz.su systemd[1]: oryx-speed.service: main process exited, code=exited, status=1/FAILURE Sep 06 12:00:00 htm-psycho-401.zxz.su systemd[1]: Unit oryx-speed.service entered failed state. Sep 06 12:00:00 htm-psycho-401.zxz.su systemd[1]: oryx-speed.service failed.

Sep 07 01:10:15 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:15 INFO TaskSetManager:54 - Starting task 28.0 in stage 25.0 (TID 361, htm-psycho-401.zxz.su, executor 1, partition 28, NODE_LOCAL, 7770 bytes) Sep 07 01:10:15 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:15 INFO TaskSetManager:54 - Finished task 27.0 in stage 25.0 (TID 360) in 1064 ms on htm-psycho-401.zxz.su (executor 1) (28/32) Sep 07 01:10:15 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:15 INFO BlockManagerInfo:54 - Removed taskresult_360 on htm-psycho-401.zxz.su:33540 in memory (size: 8.0 MB, free: 322.2 MB) Sep 07 01:10:16 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:16 INFO BlockManagerInfo:54 - Added taskresult_361 in memory on htm-psycho-401.zxz.su:33540 (size: 7.0 MB, free: 315.1 MB) Sep 07 01:10:16 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:16 INFO TaskSetManager:54 - Starting task 29.0 in stage 25.0 (TID 362, htm-psycho-401.zxz.su, executor 1, partition 29, NODE_LOCAL, 7770 bytes) Sep 07 01:10:18 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:18 INFO BlockManagerInfo:54 - Added taskresult_362 in memory on htm-psycho-401.zxz.su:33540 (size: 6.5 MB, free: 308.6 MB) Sep 07 01:10:18 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:18 INFO TaskSetManager:54 - Marking task 29 in stage 25.0 (on htm-psycho-401.zxz.su) as speculatable because it ran more than 1516 ms Sep 07 01:10:18 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:18 INFO TaskSetManager:54 - Starting task 30.0 in stage 25.0 (TID 363, htm-psycho-401.zxz.su, executor 1, partition 30, NODE_LOCAL, 7770 bytes) Sep 07 01:10:18 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:18 INFO TaskSetManager:54 - Finished task 28.0 in stage 25.0 (TID 361) in 3200 ms on htm-psycho-401.zxz.su (executor 1) (29/32) Sep 07 01:10:18 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:18 INFO BlockManagerInfo:54 - Removed taskresult_361 on htm-psycho-401.zxz.su:33540 in memory (size: 7.0 MB, free: 315.7 MB) Sep 07 01:10:18 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:18 INFO TaskSetManager:54 - Finished task 29.0 in stage 25.0 (TID 362) in 2392 ms on htm-psycho-401.zxz.su (executor 1) (30/32) Sep 07 01:10:18 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:18 INFO BlockManagerInfo:54 - Removed taskresult_362 on htm-psycho-401.zxz.su:33540 in memory (size: 6.5 MB, free: 322.2 MB) Sep 07 01:10:19 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:19 INFO BlockManagerInfo:54 - Added taskresult_363 in memory on htm-psycho-401.zxz.su:33540 (size: 7.0 MB, free: 315.2 MB) Sep 07 01:10:19 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:19 INFO TaskSetManager:54 - Starting task 31.0 in stage 25.0 (TID 364, htm-psycho-401.zxz.su, executor 1, partition 31, NODE_LOCAL, 7770 bytes) Sep 07 01:10:19 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:19 INFO TaskSetManager:54 - Finished task 30.0 in stage 25.0 (TID 363) in 892 ms on htm-psycho-401.zxz.su (executor 1) (31/32) Sep 07 01:10:19 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:19 INFO BlockManagerInfo:54 - Removed taskresult_363 on htm-psycho-401.zxz.su:33540 in memory (size: 7.0 MB, free: 322.2 MB) Sep 07 01:10:20 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:20 INFO BlockManagerInfo:54 - Added taskresult_364 in memory on htm-psycho-401.zxz.su:33540 (size: 8.9 MB, free: 313.2 MB) Sep 07 01:10:20 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:20 INFO TaskSetManager:54 - Finished task 31.0 in stage 25.0 (TID 364) in 1283 ms on htm-psycho-401.zxz.su (executor 1) (32/32) Sep 07 01:10:20 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:20 INFO YarnScheduler:54 - Removed TaskSet 25.0, whose tasks have all completed, from pool Sep 07 01:10:20 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:20 INFO DAGScheduler:54 - ResultStage 25 (collectAsMap at ALSUpdate.java:187) finished in 32.455 s Sep 07 01:10:20 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:20 INFO DAGScheduler:54 - Job 15 finished: collectAsMap at ALSUpdate.java:187, took 32.461504 s Sep 07 01:10:20 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:20 INFO BlockManagerInfo:54 - Removed taskresult_364 on htm-psycho-401.zxz.su:33540 in memory (size: 8.9 MB, free: 322.2 MB) Sep 07 01:10:26 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:26 INFO ContextCleaner:54 - Cleaned accumulator 489 Sep 07 01:10:26 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:26 INFO ContextCleaner:54 - Cleaned accumulator 493 Sep 07 01:10:26 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:26 INFO ContextCleaner:54 - Cleaned accumulator 497 Sep 07 01:10:26 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:26 INFO ContextCleaner:54 - Cleaned accumulator 490 Sep 07 01:10:26 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:26 INFO ContextCleaner:54 - Cleaned accumulator 480 Sep 07 01:10:26 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:26 INFO ContextCleaner:54 - Cleaned accumulator 496 Sep 07 01:10:26 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:26 INFO ContextCleaner:54 - Cleaned accumulator 486 Sep 07 01:10:26 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:26 INFO ContextCleaner:54 - Cleaned accumulator 477 Sep 07 01:10:26 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:26 INFO ContextCleaner:54 - Cleaned accumulator 492 Sep 07 01:10:26 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:26 INFO ContextCleaner:54 - Cleaned accumulator 481 Sep 07 01:10:26 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:26 INFO BlockManagerInfo:54 - Removed broadcast_20_piece0 on htm-psycho-401.zxz.su:39017 in memory (size: 4.1 KB, free: 912.3 MB) Sep 07 01:10:26 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:26 INFO BlockManagerInfo:54 - Removed broadcast_20_piece0 on htm-psycho-401.zxz.su:33540 in memory (size: 4.1 KB, free: 322.2 MB) Sep 07 01:10:26 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:26 INFO ContextCleaner:54 - Cleaned accumulator 482 Sep 07 01:10:26 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:26 INFO ContextCleaner:54 - Cleaned accumulator 479 Sep 07 01:10:26 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:26 INFO ContextCleaner:54 - Cleaned accumulator 476 Sep 07 01:10:26 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:26 INFO ContextCleaner:54 - Cleaned accumulator 498 Sep 07 01:10:26 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:26 INFO ContextCleaner:54 - Cleaned accumulator 487 Sep 07 01:10:26 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:26 INFO ContextCleaner:54 - Cleaned accumulator 494 Sep 07 01:10:26 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:26 INFO ContextCleaner:54 - Cleaned accumulator 491 Sep 07 01:10:26 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:26 INFO ContextCleaner:54 - Cleaned accumulator 485 Sep 07 01:10:26 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:26 INFO ContextCleaner:54 - Cleaned accumulator 495 Sep 07 01:10:26 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:26 INFO ContextCleaner:54 - Cleaned accumulator 483 Sep 07 01:10:26 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:26 INFO ContextCleaner:54 - Cleaned accumulator 478 Sep 07 01:10:26 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:26 INFO ContextCleaner:54 - Cleaned accumulator 499 Sep 07 01:10:26 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:26 INFO ContextCleaner:54 - Cleaned accumulator 475 Sep 07 01:10:26 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:26 INFO ContextCleaner:54 - Cleaned accumulator 484 Sep 07 01:10:26 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:26 INFO ContextCleaner:54 - Cleaned accumulator 488 Sep 07 01:10:42 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:42 INFO ContextCleaner:54 - Cleaned shuffle 2 Sep 07 01:10:42 htm-psycho-401.zxz.su bash[9560]: 2018-09-07 01:10:42 INFO ContextCleaner:54 - Cleaned shuffle 3

Sep 07 01:10:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:10:00 INFO ZkEventThread:65 - Starting ZkClient event thread. Sep 07 01:10:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:10:00 INFO ZkClient:936 - Waiting for keeper state SyncConnected Sep 07 01:10:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:10:00 INFO ClientCnxn:975 - Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) Sep 07 01:10:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:10:00 INFO ClientCnxn:852 - Socket connection established to 127.0.0.1/127.0.0.1:2181, initiating session Sep 07 01:10:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:10:00 INFO ClientCnxn:1235 - Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x10160d98a7901ef, negotiated timeout = 30000 Sep 07 01:10:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:10:00 INFO ZkClient:713 - zookeeper state changed (SyncConnected) Sep 07 01:10:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:10:00 INFO ZkEventThread:83 - Terminate ZkClient event thread. Sep 07 01:10:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:10:00 INFO ClientCnxn:512 - EventThread shut down Sep 07 01:10:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:10:00 INFO ZooKeeper:684 - Session: 0x10160d98a7901ef closed Sep 07 01:10:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:10:00 INFO JobScheduler:54 - Finished job streaming job 1536271800000 ms.1 from job set of time 1536271800000 ms Sep 07 01:10:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:10:00 INFO MapPartitionsRDD:54 - Removing RDD 3 from persistence list Sep 07 01:10:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:10:00 INFO JobScheduler:54 - Total delay: 0.196 s for time 1536271800000 ms (execution: 0.188 s) Sep 07 01:10:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:10:00 INFO KafkaRDD:54 - Removing RDD 2 from persistence list Sep 07 01:10:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:10:00 INFO BlockManager:54 - Removing RDD 3 Sep 07 01:10:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:10:00 INFO ReceivedBlockTracker:54 - Deleting batches: Sep 07 01:10:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:10:00 INFO InputInfoTracker:54 - remove old batch metadata: 1536271200000 ms Sep 07 01:10:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:10:00 INFO BlockManager:54 - Removing RDD 2

Sep 07 01:14:16 htm-psycho-401.zxz.su bash[9560]: at scala.collection.generic.GenTraversableFactory$GenericCanBuildFrom.apply(GenTr aversableFactory.scala:57) Sep 07 01:14:16 htm-psycho-401.zxz.su bash[9560]: at scala.collection.generic.GenTraversableFactory$GenericCanBuildFrom.apply(GenTr aversableFactory.scala:52) Sep 07 01:14:16 htm-psycho-401.zxz.su bash[9560]: at scala.collection.TraversableLike$class.builder$1(TraversableLike.scala:229) Sep 07 01:14:16 htm-psycho-401.zxz.su bash[9560]: at scala.collection.TraversableLike$class.map(TraversableLike.scala:233) Sep 07 01:14:16 htm-psycho-401.zxz.su bash[9560]: at scala.collection.AbstractTraversable.map(Traversable.scala:104) Sep 07 01:14:16 htm-psycho-401.zxz.su bash[9560]: at org.apache.spark.scheduler.TaskSchedulerImpl.resourceOffers(TaskSchedulerImpl. scala:348) Sep 07 01:14:16 htm-psycho-401.zxz.su bash[9560]: at org.apache.spark.scheduler.cluster.CoarseGrainedSchedulerBackend$DriverEndpoin t.org$apache$spark$scheduler$cluster$CoarseGrainedSchedulerBackend$DriverEndpoint$$makeOffers(CoarseGrainedSchedulerBackend.scala:2 47) Sep 07 01:14:16 htm-psycho-401.zxz.su bash[9560]: at org.apache.spark.scheduler.cluster.CoarseGrainedSchedulerBackend$DriverEndpoin t$$anonfun$receive$1.applyOrElse(CoarseGrainedSchedulerBackend.scala:136) Sep 07 01:14:16 htm-psycho-401.zxz.su bash[9560]: at org.apache.spark.rpc.netty.Inbox$$anonfun$process$1.apply$mcV$sp(Inbox.scala:1 17) Sep 07 01:14:16 htm-psycho-401.zxz.su bash[9560]: at org.apache.spark.rpc.netty.Inbox.safelyCall(Inbox.scala:205) Sep 07 01:14:16 htm-psycho-401.zxz.su bash[9560]: at org.apache.spark.rpc.netty.Inbox.process(Inbox.scala:101) Sep 07 01:14:16 htm-psycho-401.zxz.su bash[9560]: at org.apache.spark.rpc.netty.Dispatcher$MessageLoop.run(Dispatcher.scala:221) Sep 07 01:14:16 htm-psycho-401.zxz.su bash[9560]: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149 ) Sep 07 01:14:16 htm-psycho-401.zxz.su bash[9560]: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624 ) Sep 07 01:14:16 htm-psycho-401.zxz.su bash[9560]: at java.lang.Thread.run(Thread.java:748) Sep 07 01:15:17 htm-psycho-401.zxz.su bash[9560]: Exception in thread "streaming-job-executor-0" java.lang.OutOfMemoryError: GC ove rhead limit exceeded

I've added more memory:

batch {
    streaming {
      generation-interval-sec = 300
      num-executors = 1
      executor-cores = 1
      executor-memory = "4g"
      driver-memory = "3g"
      config = {
             spark.kryoserializer.buffer.max = "2047m"
      }
    }
}

It doesn't seem to crash, but I still get result 503 after uploading csv file. batch:

Sep 07 01:45:34 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:34 INFO  TaskSetManager:54 - Finished task 10.0 in stage 21.0 (TID 335) in 3977 ms on htm-psycho-401.zxz.su (executor 1) (11/32)
Sep 07 01:45:34 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:34 INFO  BlockManagerInfo:54 - Removed taskresult_335 on htm-psycho-401.zxz.su:41761 in memory (size: 8.0 MB, free: 175.4 MB)
Sep 07 01:45:34 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:34 INFO  TaskSetManager:54 - Finished task 11.0 in stage 21.0 (TID 336) in 3062 ms on htm-psycho-401.zxz.su (executor 1) (12/32)
Sep 07 01:45:34 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:34 INFO  BlockManagerInfo:54 - Removed taskresult_336 on htm-psycho-401.zxz.su:41761 in memory (size: 6.3 MB, free: 181.7 MB)
Sep 07 01:45:35 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:35 INFO  BlockManagerInfo:54 - Added taskresult_337 in memory on htm-psycho-401.zxz.su:41761 (size: 7.4 MB, free: 174.3 MB)
Sep 07 01:45:35 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:35 INFO  TaskSetManager:54 - Starting task 13.0 in stage 21.0 (TID 338, htm-psycho-401.zxz.su, executor 1, partition 13, NODE_LOCAL, 7770 bytes)
Sep 07 01:45:35 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:35 INFO  TaskSetManager:54 - Finished task 12.0 in stage 21.0 (TID 337) in 966 ms on htm-psycho-401.zxz.su (executor 1) (13/32)
Sep 07 01:45:35 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:35 INFO  BlockManagerInfo:54 - Removed taskresult_337 on htm-psycho-401.zxz.su:41761 in memory (size: 7.4 MB, free: 181.7 MB)
Sep 07 01:45:36 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:36 INFO  BlockManagerInfo:54 - Added taskresult_338 in memory on htm-psycho-401.zxz.su:41761 (size: 7.7 MB, free: 174.0 MB)
Sep 07 01:45:36 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:36 INFO  TaskSetManager:54 - Starting task 14.0 in stage 21.0 (TID 339, htm-psycho-401.zxz.su, executor 1, partition 14, NODE_LOCAL, 7770 bytes)
Sep 07 01:45:36 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:36 INFO  TaskSetManager:54 - Finished task 13.0 in stage 21.0 (TID 338) in 1059 ms on htm-psycho-401.zxz.su (executor 1) (14/32)
Sep 07 01:45:36 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:36 INFO  BlockManagerInfo:54 - Removed taskresult_338 on htm-psycho-401.zxz.su:41761 in memory (size: 7.7 MB, free: 181.7 MB)
Sep 07 01:45:36 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:36 INFO  BlockManagerInfo:54 - Added taskresult_339 in memory on htm-psycho-401.zxz.su:41761 (size: 6.3 MB, free: 175.4 MB)
Sep 07 01:45:36 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:36 INFO  TaskSetManager:54 - Starting task 15.0 in stage 21.0 (TID 340, htm-psycho-401.zxz.su, executor 1, partition 15, NODE_LOCAL, 7770 bytes)
Sep 07 01:45:36 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:36 INFO  TaskSetManager:54 - Finished task 14.0 in stage 21.0 (TID 339) in 823 ms on htm-psycho-401.zxz.su (executor 1) (15/32)
Sep 07 01:45:36 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:36 INFO  BlockManagerInfo:54 - Removed taskresult_339 on htm-psycho-401.zxz.su:41761 in memory (size: 6.3 MB, free: 181.7 MB)
Sep 07 01:45:37 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:37 INFO  BlockManagerInfo:54 - Added taskresult_340 in memory on htm-psycho-401.zxz.su:41761 (size: 8.4 MB, free: 173.2 MB)
Sep 07 01:45:37 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:37 INFO  TaskSetManager:54 - Starting task 16.0 in stage 21.0 (TID 341, htm-psycho-401.zxz.su, executor 1, partition 16, NODE_LOCAL, 7770 bytes)
Sep 07 01:45:38 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:38 INFO  TaskSetManager:54 - Finished task 15.0 in stage 21.0 (TID 340) in 1194 ms on htm-psycho-401.zxz.su (executor 1) (16/32)
Sep 07 01:45:38 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:38 INFO  BlockManagerInfo:54 - Removed taskresult_340 on htm-psycho-401.zxz.su:41761 in memory (size: 8.4 MB, free: 181.7 MB)
Sep 07 01:45:38 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:38 INFO  BlockManagerInfo:54 - Added taskresult_341 in memory on htm-psycho-401.zxz.su:41761 (size: 7.3 MB, free: 174.4 MB)
Sep 07 01:45:38 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:38 INFO  TaskSetManager:54 - Starting task 17.0 in stage 21.0 (TID 342, htm-psycho-401.zxz.su, executor 1, partition 17, NODE_LOCAL, 7770 bytes)
Sep 07 01:45:38 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:38 INFO  TaskSetManager:54 - Finished task 16.0 in stage 21.0 (TID 341) in 951 ms on htm-psycho-401.zxz.su (executor 1) (17/32)
Sep 07 01:45:38 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:38 INFO  BlockManagerInfo:54 - Removed taskresult_341 on htm-psycho-401.zxz.su:41761 in memory (size: 7.3 MB, free: 181.7 MB)
Sep 07 01:45:39 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:39 INFO  BlockManagerInfo:54 - Added taskresult_342 in memory on htm-psycho-401.zxz.su:41761 (size: 6.3 MB, free: 175.4 MB)
Sep 07 01:45:39 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:39 INFO  TaskSetManager:54 - Starting task 18.0 in stage 21.0 (TID 343, htm-psycho-401.zxz.su, executor 1, partition 18, NODE_LOCAL, 7770 bytes)
Sep 07 01:45:39 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:39 INFO  TaskSetManager:54 - Finished task 17.0 in stage 21.0 (TID 342) in 947 ms on htm-psycho-401.zxz.su (executor 1) (18/32)
Sep 07 01:45:39 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:39 INFO  BlockManagerInfo:54 - Removed taskresult_342 on htm-psycho-401.zxz.su:41761 in memory (size: 6.3 MB, free: 181.7 MB)
Sep 07 01:45:40 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:40 INFO  BlockManagerInfo:54 - Added taskresult_343 in memory on htm-psycho-401.zxz.su:41761 (size: 8.0 MB, free: 173.7 MB)
Sep 07 01:45:40 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:40 INFO  TaskSetManager:54 - Starting task 19.0 in stage 21.0 (TID 344, htm-psycho-401.zxz.su, executor 1, partition 19, NODE_LOCAL, 7770 bytes)
Sep 07 01:45:40 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:40 INFO  TaskSetManager:54 - Finished task 18.0 in stage 21.0 (TID 343) in 1131 ms on htm-psycho-401.zxz.su (executor 1) (19/32)
Sep 07 01:45:40 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:40 INFO  BlockManagerInfo:54 - Removed taskresult_343 on htm-psycho-401.zxz.su:41761 in memory (size: 8.0 MB, free: 181.7 MB)
Sep 07 01:45:41 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:41 INFO  BlockManagerInfo:54 - Added taskresult_344 in memory on htm-psycho-401.zxz.su:41761 (size: 7.7 MB, free: 174.0 MB)
Sep 07 01:45:41 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:41 INFO  TaskSetManager:54 - Starting task 20.0 in stage 21.0 (TID 345, htm-psycho-401.zxz.su, executor 1, partition 20, NODE_LOCAL, 7770 bytes)
Sep 07 01:45:41 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:41 INFO  TaskSetManager:54 - Finished task 19.0 in stage 21.0 (TID 344) in 1006 ms on htm-psycho-401.zxz.su (executor 1) (20/32)
Sep 07 01:45:41 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:41 INFO  BlockManagerInfo:54 - Removed taskresult_344 on htm-psycho-401.zxz.su:41761 in memory (size: 7.7 MB, free: 181.7 MB)
Sep 07 01:45:42 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:42 INFO  BlockManagerInfo:54 - Added taskresult_345 in memory on htm-psycho-401.zxz.su:41761 (size: 5.5 MB, free: 176.2 MB)
Sep 07 01:45:42 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:42 INFO  TaskSetManager:54 - Starting task 21.0 in stage 21.0 (TID 346, htm-psycho-401.zxz.su, executor 1, partition 21, NODE_LOCAL, 7770 bytes)
Sep 07 01:45:42 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:42 INFO  TaskSetManager:54 - Finished task 20.0 in stage 21.0 (TID 345) in 752 ms on htm-psycho-401.zxz.su (executor 1) (21/32)
Sep 07 01:45:42 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:42 INFO  BlockManagerInfo:54 - Removed taskresult_345 on htm-psycho-401.zxz.su:41761 in memory (size: 5.5 MB, free: 181.7 MB)
Sep 07 01:45:43 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:43 INFO  BlockManagerInfo:54 - Added taskresult_346 in memory on htm-psycho-401.zxz.su:41761 (size: 9.0 MB, free: 172.7 MB)
Sep 07 01:45:43 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:43 INFO  TaskSetManager:54 - Starting task 22.0 in stage 21.0 (TID 347, htm-psycho-401.zxz.su, executor 1, partition 22, NODE_LOCAL, 7770 bytes)
Sep 07 01:45:43 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:43 INFO  TaskSetManager:54 - Finished task 21.0 in stage 21.0 (TID 346) in 1273 ms on htm-psycho-401.zxz.su (executor 1) (22/32)
Sep 07 01:45:43 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:43 INFO  BlockManagerInfo:54 - Removed taskresult_346 on htm-psycho-401.zxz.su:41761 in memory (size: 9.0 MB, free: 181.7 MB)
Sep 07 01:45:44 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:44 INFO  BlockManagerInfo:54 - Added taskresult_347 in memory on htm-psycho-401.zxz.su:41761 (size: 7.1 MB, free: 174.6 MB)
Sep 07 01:45:44 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:44 INFO  TaskSetManager:54 - Starting task 23.0 in stage 21.0 (TID 348, htm-psycho-401.zxz.su, executor 1, partition 23, NODE_LOCAL, 7770 bytes)
Sep 07 01:45:44 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:44 INFO  TaskSetManager:54 - Finished task 22.0 in stage 21.0 (TID 347) in 927 ms on htm-psycho-401.zxz.su (executor 1) (23/32)
Sep 07 01:45:44 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:44 INFO  BlockManagerInfo:54 - Removed taskresult_347 on htm-psycho-401.zxz.su:41761 in memory (size: 7.1 MB, free: 181.7 MB)
Sep 07 01:45:45 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:45 INFO  BlockManagerInfo:54 - Added taskresult_348 in memory on htm-psycho-401.zxz.su:41761 (size: 8.2 MB, free: 173.4 MB)
Sep 07 01:45:45 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:45 INFO  TaskSetManager:54 - Starting task 24.0 in stage 21.0 (TID 349, htm-psycho-401.zxz.su, executor 1, partition 24, NODE_LOCAL, 7770 bytes)
Sep 07 01:45:45 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:45 INFO  TaskSetManager:54 - Finished task 23.0 in stage 21.0 (TID 348) in 1615 ms on htm-psycho-401.zxz.su (executor 1) (24/32)
Sep 07 01:45:45 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:45 INFO  BlockManagerInfo:54 - Removed taskresult_348 on htm-psycho-401.zxz.su:41761 in memory (size: 8.2 MB, free: 181.7 MB)
Sep 07 01:45:46 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:46 INFO  BlockManagerInfo:54 - Added taskresult_349 in memory on htm-psycho-401.zxz.su:41761 (size: 5.9 MB, free: 175.7 MB)
Sep 07 01:45:46 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:46 INFO  TaskSetManager:54 - Starting task 25.0 in stage 21.0 (TID 350, htm-psycho-401.zxz.su, executor 1, partition 25, NODE_LOCAL, 7770 bytes)
Sep 07 01:45:46 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:46 INFO  TaskSetManager:54 - Finished task 24.0 in stage 21.0 (TID 349) in 943 ms on htm-psycho-401.zxz.su (executor 1) (25/32)
Sep 07 01:45:46 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:46 INFO  BlockManagerInfo:54 - Removed taskresult_349 on htm-psycho-401.zxz.su:41761 in memory (size: 5.9 MB, free: 181.7 MB)
Sep 07 01:45:46 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:46 INFO  BlockManagerInfo:54 - Added taskresult_350 in memory on htm-psycho-401.zxz.su:41761 (size: 6.7 MB, free: 175.0 MB)
Sep 07 01:45:46 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:46 INFO  TaskSetManager:54 - Starting task 26.0 in stage 21.0 (TID 351, htm-psycho-401.zxz.su, executor 1, partition 26, NODE_LOCAL, 7770 bytes)
Sep 07 01:45:47 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:47 INFO  TaskSetManager:54 - Finished task 25.0 in stage 21.0 (TID 350) in 864 ms on htm-psycho-401.zxz.su (executor 1) (26/32)
Sep 07 01:45:47 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:47 INFO  BlockManagerInfo:54 - Removed taskresult_350 on htm-psycho-401.zxz.su:41761 in memory (size: 6.7 MB, free: 181.7 MB)
Sep 07 01:45:48 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:48 INFO  BlockManagerInfo:54 - Added taskresult_351 in memory on htm-psycho-401.zxz.su:41761 (size: 8.4 MB, free: 173.3 MB)
Sep 07 01:45:48 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:48 INFO  TaskSetManager:54 - Starting task 27.0 in stage 21.0 (TID 352, htm-psycho-401.zxz.su, executor 1, partition 27, NODE_LOCAL, 7770 bytes)
Sep 07 01:45:48 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:48 INFO  TaskSetManager:54 - Finished task 26.0 in stage 21.0 (TID 351) in 1196 ms on htm-psycho-401.zxz.su (executor 1) (27/32)
Sep 07 01:45:48 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:48 INFO  BlockManagerInfo:54 - Removed taskresult_351 on htm-psycho-401.zxz.su:41761 in memory (size: 8.4 MB, free: 181.7 MB)
Sep 07 01:45:48 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:48 INFO  BlockManagerInfo:54 - Added taskresult_352 in memory on htm-psycho-401.zxz.su:41761 (size: 7.6 MB, free: 174.0 MB)
Sep 07 01:45:48 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:48 INFO  TaskSetManager:54 - Starting task 28.0 in stage 21.0 (TID 353, htm-psycho-401.zxz.su, executor 1, partition 28, NODE_LOCAL, 7770 bytes)
Sep 07 01:45:49 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:49 INFO  TaskSetManager:54 - Finished task 27.0 in stage 21.0 (TID 352) in 966 ms on htm-psycho-401.zxz.su (executor 1) (28/32)
Sep 07 01:45:49 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:49 INFO  BlockManagerInfo:54 - Removed taskresult_352 on htm-psycho-401.zxz.su:41761 in memory (size: 7.6 MB, free: 181.7 MB)
Sep 07 01:45:49 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:49 INFO  BlockManagerInfo:54 - Added taskresult_353 in memory on htm-psycho-401.zxz.su:41761 (size: 8.0 MB, free: 173.7 MB)
Sep 07 01:45:49 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:49 INFO  TaskSetManager:54 - Starting task 29.0 in stage 21.0 (TID 354, htm-psycho-401.zxz.su, executor 1, partition 29, NODE_LOCAL, 7770 bytes)
Sep 07 01:45:50 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:50 INFO  TaskSetManager:54 - Finished task 28.0 in stage 21.0 (TID 353) in 1042 ms on htm-psycho-401.zxz.su (executor 1) (29/32)
Sep 07 01:45:50 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:50 INFO  BlockManagerInfo:54 - Removed taskresult_353 on htm-psycho-401.zxz.su:41761 in memory (size: 8.0 MB, free: 181.7 MB)
Sep 07 01:45:51 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:51 INFO  BlockManagerInfo:54 - Added taskresult_354 in memory on htm-psycho-401.zxz.su:41761 (size: 9.2 MB, free: 172.5 MB)
Sep 07 01:45:51 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:51 INFO  TaskSetManager:54 - Starting task 30.0 in stage 21.0 (TID 355, htm-psycho-401.zxz.su, executor 1, partition 30, NODE_LOCAL, 7770 bytes)
Sep 07 01:45:51 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:51 INFO  TaskSetManager:54 - Finished task 29.0 in stage 21.0 (TID 354) in 1305 ms on htm-psycho-401.zxz.su (executor 1) (30/32)
Sep 07 01:45:51 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:51 INFO  BlockManagerInfo:54 - Removed taskresult_354 on htm-psycho-401.zxz.su:41761 in memory (size: 9.2 MB, free: 181.7 MB)
Sep 07 01:45:51 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:51 INFO  BlockManagerInfo:54 - Added taskresult_355 in memory on htm-psycho-401.zxz.su:41761 (size: 6.0 MB, free: 175.7 MB)
Sep 07 01:45:51 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:51 INFO  TaskSetManager:54 - Starting task 31.0 in stage 21.0 (TID 356, htm-psycho-401.zxz.su, executor 1, partition 31, NODE_LOCAL, 7770 bytes)
Sep 07 01:45:54 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:54 INFO  TaskSetManager:54 - Marking task 31 in stage 21.0 (on htm-psycho-401.zxz.su) as speculatable because it ran more than 1479 ms
Sep 07 01:45:54 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:54 INFO  BlockManagerInfo:54 - Added taskresult_356 in memory on htm-psycho-401.zxz.su:41761 (size: 6.5 MB, free: 169.2 MB)
Sep 07 01:45:54 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:54 INFO  TaskSetManager:54 - Finished task 30.0 in stage 21.0 (TID 355) in 3564 ms on htm-psycho-401.zxz.su (executor 1) (31/32)
Sep 07 01:45:54 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:54 INFO  BlockManagerInfo:54 - Removed taskresult_355 on htm-psycho-401.zxz.su:41761 in memory (size: 6.0 MB, free: 175.2 MB)
Sep 07 01:45:54 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:54 INFO  TaskSetManager:54 - Finished task 31.0 in stage 21.0 (TID 356) in 2894 ms on htm-psycho-401.zxz.su (executor 1) (32/32)
Sep 07 01:45:54 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:54 INFO  YarnScheduler:54 - Removed TaskSet 21.0, whose tasks have all completed, from pool
Sep 07 01:45:54 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:54 INFO  DAGScheduler:54 - ResultStage 21 (collectAsMap at ALSUpdate.java:187) finished in 33.544 s
Sep 07 01:45:54 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:54 INFO  DAGScheduler:54 - Job 11 finished: collectAsMap at ALSUpdate.java:187, took 33.551173 s
Sep 07 01:45:54 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:45:54 INFO  BlockManagerInfo:54 - Removed taskresult_356 on htm-psycho-401.zxz.su:41761 in memory (size: 6.5 MB, free: 181.7 MB)
Sep 07 01:46:07 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:07 INFO  ALSUpdate:133 - Broadcasting ID-index mappings for 2789778 users, 5998833 items
Sep 07 01:46:07 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:07 INFO  MemoryStore:54 - Block broadcast_17 stored as values in memory (estimated size 393.0 MB, free 1065.3 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  MemoryStore:54 - Block broadcast_17_piece0 stored as bytes in memory (estimated size 4.0 MB, free 1061.3 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManagerInfo:54 - Added broadcast_17_piece0 in memory on htm-psycho-401.zxz.su:42625 (size: 4.0 MB, free: 1454.6 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  MemoryStore:54 - Block broadcast_17_piece1 stored as bytes in memory (estimated size 4.0 MB, free 1057.3 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManagerInfo:54 - Added broadcast_17_piece1 in memory on htm-psycho-401.zxz.su:42625 (size: 4.0 MB, free: 1450.6 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  MemoryStore:54 - Block broadcast_17_piece2 stored as bytes in memory (estimated size 4.0 MB, free 1053.3 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManagerInfo:54 - Added broadcast_17_piece2 in memory on htm-psycho-401.zxz.su:42625 (size: 4.0 MB, free: 1446.6 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  MemoryStore:54 - Block broadcast_17_piece3 stored as bytes in memory (estimated size 4.0 MB, free 1049.3 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManagerInfo:54 - Added broadcast_17_piece3 in memory on htm-psycho-401.zxz.su:42625 (size: 4.0 MB, free: 1442.6 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  MemoryStore:54 - Block broadcast_17_piece4 stored as bytes in memory (estimated size 4.0 MB, free 1045.3 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManagerInfo:54 - Added broadcast_17_piece4 in memory on htm-psycho-401.zxz.su:42625 (size: 4.0 MB, free: 1438.6 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  MemoryStore:54 - Block broadcast_17_piece5 stored as bytes in memory (estimated size 4.0 MB, free 1041.3 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManagerInfo:54 - Added broadcast_17_piece5 in memory on htm-psycho-401.zxz.su:42625 (size: 4.0 MB, free: 1434.6 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  MemoryStore:54 - Block broadcast_17_piece6 stored as bytes in memory (estimated size 4.0 MB, free 1037.3 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManagerInfo:54 - Added broadcast_17_piece6 in memory on htm-psycho-401.zxz.su:42625 (size: 4.0 MB, free: 1430.6 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  MemoryStore:54 - Block broadcast_17_piece7 stored as bytes in memory (estimated size 4.0 MB, free 1033.3 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManagerInfo:54 - Added broadcast_17_piece7 in memory on htm-psycho-401.zxz.su:42625 (size: 4.0 MB, free: 1426.6 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  MemoryStore:54 - Block broadcast_17_piece8 stored as bytes in memory (estimated size 4.0 MB, free 1029.3 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManagerInfo:54 - Added broadcast_17_piece8 in memory on htm-psycho-401.zxz.su:42625 (size: 4.0 MB, free: 1422.6 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  MemoryStore:54 - Block broadcast_17_piece9 stored as bytes in memory (estimated size 4.0 MB, free 1025.3 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManagerInfo:54 - Added broadcast_17_piece9 in memory on htm-psycho-401.zxz.su:42625 (size: 4.0 MB, free: 1418.6 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  MemoryStore:54 - Block broadcast_17_piece10 stored as bytes in memory (estimated size 4.0 MB, free 1021.3 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManagerInfo:54 - Added broadcast_17_piece10 in memory on htm-psycho-401.zxz.su:42625 (size: 4.0 MB, free: 1414.6 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  MemoryStore:54 - Block broadcast_17_piece11 stored as bytes in memory (estimated size 4.0 MB, free 1017.3 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManagerInfo:54 - Added broadcast_17_piece11 in memory on htm-psycho-401.zxz.su:42625 (size: 4.0 MB, free: 1410.6 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  MemoryStore:54 - Block broadcast_17_piece12 stored as bytes in memory (estimated size 4.0 MB, free 1013.3 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManagerInfo:54 - Added broadcast_17_piece12 in memory on htm-psycho-401.zxz.su:42625 (size: 4.0 MB, free: 1406.6 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  MemoryStore:54 - Block broadcast_17_piece13 stored as bytes in memory (estimated size 4.0 MB, free 1009.3 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManagerInfo:54 - Added broadcast_17_piece13 in memory on htm-psycho-401.zxz.su:42625 (size: 4.0 MB, free: 1402.6 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  MemoryStore:54 - Block broadcast_17_piece14 stored as bytes in memory (estimated size 4.0 MB, free 1005.3 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManagerInfo:54 - Added broadcast_17_piece14 in memory on htm-psycho-401.zxz.su:42625 (size: 4.0 MB, free: 1398.6 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  MemoryStore:54 - Block broadcast_17_piece15 stored as bytes in memory (estimated size 4.0 MB, free 1001.3 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManagerInfo:54 - Added broadcast_17_piece15 in memory on htm-psycho-401.zxz.su:42625 (size: 4.0 MB, free: 1394.6 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  MemoryStore:54 - Block broadcast_17_piece16 stored as bytes in memory (estimated size 4.0 MB, free 997.3 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManagerInfo:54 - Added broadcast_17_piece16 in memory on htm-psycho-401.zxz.su:42625 (size: 4.0 MB, free: 1390.6 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  MemoryStore:54 - Block broadcast_17_piece17 stored as bytes in memory (estimated size 4.0 MB, free 993.3 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManagerInfo:54 - Added broadcast_17_piece17 in memory on htm-psycho-401.zxz.su:42625 (size: 4.0 MB, free: 1386.6 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  MemoryStore:54 - Block broadcast_17_piece18 stored as bytes in memory (estimated size 4.0 MB, free 989.3 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManagerInfo:54 - Added broadcast_17_piece18 in memory on htm-psycho-401.zxz.su:42625 (size: 4.0 MB, free: 1382.6 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  MemoryStore:54 - Block broadcast_17_piece19 stored as bytes in memory (estimated size 4.0 MB, free 985.3 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManagerInfo:54 - Added broadcast_17_piece19 in memory on htm-psycho-401.zxz.su:42625 (size: 4.0 MB, free: 1378.6 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  MemoryStore:54 - Block broadcast_17_piece20 stored as bytes in memory (estimated size 4.0 MB, free 981.3 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManagerInfo:54 - Added broadcast_17_piece20 in memory on htm-psycho-401.zxz.su:42625 (size: 4.0 MB, free: 1374.6 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  MemoryStore:54 - Block broadcast_17_piece21 stored as bytes in memory (estimated size 1207.3 KB, free 980.1 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManagerInfo:54 - Added broadcast_17_piece21 in memory on htm-psycho-401.zxz.su:42625 (size: 1207.3 KB, free: 1373.4 MB)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  SparkContext:54 - Created broadcast 17 from broadcast at ALSUpdate.java:136
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  MemoryStore:54 - 5 blocks selected for dropping (393.3 MB bytes)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManager:54 - Dropping block broadcast_1 from memory
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManager:54 - Writing block broadcast_1 to disk
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManager:54 - Dropping block broadcast_1_piece0 from memory
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManager:54 - Writing block broadcast_1_piece0 to disk
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManagerInfo:54 - Updated broadcast_1_piece0 on disk on htm-psycho-401.zxz.su:42625 (current size: 23.5 KB, original size: 0.0 B)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManager:54 - Dropping block broadcast_16 from memory
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManager:54 - Writing block broadcast_16 to disk
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManager:54 - Dropping block broadcast_16_piece0 from memory
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManager:54 - Writing block broadcast_16_piece0 to disk
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManagerInfo:54 - Updated broadcast_16_piece0 on disk on htm-psycho-401.zxz.su:42625 (current size: 4.1 KB, original size: 0.0 B)
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManager:54 - Dropping block broadcast_17 from memory
Sep 07 01:46:12 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:12 INFO  BlockManager:54 - Writing block broadcast_17 to disk
Sep 07 01:46:19 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:19 INFO  ContextCleaner:54 - Cleaned accumulator 387
Sep 07 01:46:19 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:19 INFO  ContextCleaner:54 - Cleaned accumulator 383
Sep 07 01:46:19 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:19 INFO  ContextCleaner:54 - Cleaned accumulator 393
Sep 07 01:46:19 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:19 INFO  ContextCleaner:54 - Cleaned accumulator 392
Sep 07 01:46:19 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:19 INFO  ContextCleaner:54 - Cleaned shuffle 2
Sep 07 01:46:19 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:19 INFO  ContextCleaner:54 - Cleaned accumulator 377
Sep 07 01:46:19 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:19 INFO  ContextCleaner:54 - Cleaned accumulator 396
Sep 07 01:46:19 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:19 INFO  ContextCleaner:54 - Cleaned accumulator 385
Sep 07 01:46:19 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:19 INFO  ContextCleaner:54 - Cleaned accumulator 386
Sep 07 01:46:19 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:19 INFO  ContextCleaner:54 - Cleaned accumulator 384
Sep 07 01:46:19 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:19 INFO  ContextCleaner:54 - Cleaned accumulator 379
Sep 07 01:46:19 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:19 INFO  ContextCleaner:54 - Cleaned shuffle 3
Sep 07 01:46:19 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:19 INFO  ContextCleaner:54 - Cleaned accumulator 375
Sep 07 01:46:19 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:19 INFO  ContextCleaner:54 - Cleaned accumulator 378
Sep 07 01:46:19 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:19 INFO  ContextCleaner:54 - Cleaned accumulator 389
Sep 07 01:46:19 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:19 INFO  ContextCleaner:54 - Cleaned accumulator 399
Sep 07 01:46:19 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:19 INFO  ContextCleaner:54 - Cleaned accumulator 394
Sep 07 01:46:19 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:19 INFO  ContextCleaner:54 - Cleaned accumulator 390
Sep 07 01:46:19 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:19 INFO  ContextCleaner:54 - Cleaned accumulator 395
Sep 07 01:46:19 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:19 INFO  ContextCleaner:54 - Cleaned accumulator 397
Sep 07 01:46:19 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:46:19 INFO  BlockManagerInfo:54 - Removed broadcast_16_piece0 on htm-psycho-401.zxz.su:41761 in memory (size: 4.1 KB, free: 181.7 MB)

speed:

Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  BlockManagerInfo:54 - Added broadcast_10_piece0 in memory on htm-psycho-401.zxz.su:42317 (size: 2.4 KB, free: 912.3 MB)
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  SparkContext:54 - Created broadcast 10 from broadcast at DAGScheduler.scala:1039
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 10 (MapPartitionsRDD[13] at mapToPair at SpeedLayer.java:97) (first 15 tasks are for partitions Vector(0))
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  YarnScheduler:54 - Adding task set 10.0 with 1 tasks
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  TaskSetManager:54 - Starting task 0.0 in stage 10.0 (TID 18, htm-psycho-401.zxz.su, executor 1, partition 0, PROCESS_LOCAL, 7746 bytes)
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  BlockManagerInfo:54 - Added broadcast_10_piece0 in memory on htm-psycho-401.zxz.su:45221 (size: 2.4 KB, free: 912.3 MB)
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  TaskSetManager:54 - Finished task 0.0 in stage 10.0 (TID 18) in 32 ms on htm-psycho-401.zxz.su (executor 1) (1/1)
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  YarnScheduler:54 - Removed TaskSet 10.0, whose tasks have all completed, from pool
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  DAGScheduler:54 - ResultStage 10 (isEmpty at SpeedLayerUpdate.java:53) finished in 0.047 s
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  DAGScheduler:54 - Job 10 finished: isEmpty at SpeedLayerUpdate.java:53, took 0.050545 s
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  SparkContext:54 - Starting job: isEmpty at SpeedLayerUpdate.java:53
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  DAGScheduler:54 - Got job 11 (isEmpty at SpeedLayerUpdate.java:53) with 3 output partitions
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  DAGScheduler:54 - Final stage: ResultStage 11 (isEmpty at SpeedLayerUpdate.java:53)
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  DAGScheduler:54 - Parents of final stage: List()
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  DAGScheduler:54 - Missing parents: List()
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  DAGScheduler:54 - Submitting ResultStage 11 (MapPartitionsRDD[13] at mapToPair at SpeedLayer.java:97), which has no missing parents
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  MemoryStore:54 - Block broadcast_11 stored as values in memory (estimated size 3.8 KB, free 912.3 MB)
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  MemoryStore:54 - Block broadcast_11_piece0 stored as bytes in memory (estimated size 2.4 KB, free 912.3 MB)
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  BlockManagerInfo:54 - Added broadcast_11_piece0 in memory on htm-psycho-401.zxz.su:42317 (size: 2.4 KB, free: 912.3 MB)
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  SparkContext:54 - Created broadcast 11 from broadcast at DAGScheduler.scala:1039
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  DAGScheduler:54 - Submitting 3 missing tasks from ResultStage 11 (MapPartitionsRDD[13] at mapToPair at SpeedLayer.java:97) (first 15 tasks are for partitions Vector(1, 2, 3))
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  YarnScheduler:54 - Adding task set 11.0 with 3 tasks
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  TaskSetManager:54 - Starting task 0.0 in stage 11.0 (TID 19, htm-psycho-401.zxz.su, executor 1, partition 1, PROCESS_LOCAL, 7746 bytes)
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  BlockManagerInfo:54 - Added broadcast_11_piece0 in memory on htm-psycho-401.zxz.su:45221 (size: 2.4 KB, free: 912.3 MB)
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  TaskSetManager:54 - Starting task 1.0 in stage 11.0 (TID 20, htm-psycho-401.zxz.su, executor 1, partition 2, PROCESS_LOCAL, 7746 bytes)
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  TaskSetManager:54 - Finished task 0.0 in stage 11.0 (TID 19) in 26 ms on htm-psycho-401.zxz.su (executor 1) (1/3)
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  TaskSetManager:54 - Starting task 2.0 in stage 11.0 (TID 21, htm-psycho-401.zxz.su, executor 1, partition 3, PROCESS_LOCAL, 7746 bytes)
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  TaskSetManager:54 - Finished task 1.0 in stage 11.0 (TID 20) in 12 ms on htm-psycho-401.zxz.su (executor 1) (2/3)
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  TaskSetManager:54 - Finished task 2.0 in stage 11.0 (TID 21) in 12 ms on htm-psycho-401.zxz.su (executor 1) (3/3)
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  YarnScheduler:54 - Removed TaskSet 11.0, whose tasks have all completed, from pool
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  DAGScheduler:54 - ResultStage 11 (isEmpty at SpeedLayerUpdate.java:53) finished in 0.063 s
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  DAGScheduler:54 - Job 11 finished: isEmpty at SpeedLayerUpdate.java:53, took 0.067594 s
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  JobScheduler:54 - Finished job streaming job 1536273000000 ms.0 from job set of time 1536273000000 ms
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  JobScheduler:54 - Starting job streaming job 1536273000000 ms.1 from job set of time 1536273000000 ms
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  UpdateOffsetsFn:60 - Updating offsets: {OryxInput,0=2576784, OryxInput,1=2579068, OryxInput,2=2577833, OryxInput,3=2579317}
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  ZooKeeper:438 - Initiating client connection, connectString=127.0.0.1:2181 sessionTimeout=30000 watcher=org.I0Itec.zkclient.ZkClient@32686a1d
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  ZkEventThread:65 - Starting ZkClient event thread.
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  ZkClient:936 - Waiting for keeper state SyncConnected
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  ClientCnxn:975 - Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  ClientCnxn:852 - Socket connection established to 127.0.0.1/127.0.0.1:2181, initiating session
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  ClientCnxn:1235 - Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x10160d98a7901f3, negotiated timeout = 30000
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  ZkClient:713 - zookeeper state changed (SyncConnected)
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  ZkEventThread:83 - Terminate ZkClient event thread.
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  ZooKeeper:684 - Session: 0x10160d98a7901f3 closed
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  ClientCnxn:512 - EventThread shut down
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  JobScheduler:54 - Finished job streaming job 1536273000000 ms.1 from job set of time 1536273000000 ms
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  JobScheduler:54 - Total delay: 0.148 s for time 1536273000000 ms (execution: 0.142 s)
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  MapPartitionsRDD:54 - Removing RDD 11 from persistence list
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  BlockManager:54 - Removing RDD 11
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  KafkaRDD:54 - Removing RDD 10 from persistence list
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  BlockManager:54 - Removing RDD 10
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  ReceivedBlockTracker:54 - Deleting batches:
Sep 07 01:30:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:30:00 INFO  InputInfoTracker:54 - remove old batch metadata: 1536272400000 ms
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  JobScheduler:54 - Added jobs for time 1536273300000 ms
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  JobScheduler:54 - Starting job streaming job 1536273300000 ms.0 from job set of time 1536273300000 ms
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  SparkContext:54 - Starting job: isEmpty at SpeedLayerUpdate.java:53
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  DAGScheduler:54 - Got job 12 (isEmpty at SpeedLayerUpdate.java:53) with 1 output partitions
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  DAGScheduler:54 - Final stage: ResultStage 12 (isEmpty at SpeedLayerUpdate.java:53)
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  DAGScheduler:54 - Parents of final stage: List()
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  DAGScheduler:54 - Missing parents: List()
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  DAGScheduler:54 - Submitting ResultStage 12 (MapPartitionsRDD[15] at mapToPair at SpeedLayer.java:97), which has no missing parents
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  MemoryStore:54 - Block broadcast_12 stored as values in memory (estimated size 3.8 KB, free 912.3 MB)
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  MemoryStore:54 - Block broadcast_12_piece0 stored as bytes in memory (estimated size 2.4 KB, free 912.3 MB)
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  BlockManagerInfo:54 - Added broadcast_12_piece0 in memory on htm-psycho-401.zxz.su:42317 (size: 2.4 KB, free: 912.3 MB)
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  SparkContext:54 - Created broadcast 12 from broadcast at DAGScheduler.scala:1039
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 12 (MapPartitionsRDD[15] at mapToPair at SpeedLayer.java:97) (first 15 tasks are for partitions Vector(0))
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  YarnScheduler:54 - Adding task set 12.0 with 1 tasks
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  TaskSetManager:54 - Starting task 0.0 in stage 12.0 (TID 22, htm-psycho-401.zxz.su, executor 1, partition 0, PROCESS_LOCAL, 7746 bytes)
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  BlockManagerInfo:54 - Added broadcast_12_piece0 in memory on htm-psycho-401.zxz.su:45221 (size: 2.4 KB, free: 912.3 MB)
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  TaskSetManager:54 - Finished task 0.0 in stage 12.0 (TID 22) in 30 ms on htm-psycho-401.zxz.su (executor 1) (1/1)
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  YarnScheduler:54 - Removed TaskSet 12.0, whose tasks have all completed, from pool
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  DAGScheduler:54 - ResultStage 12 (isEmpty at SpeedLayerUpdate.java:53) finished in 0.044 s
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  DAGScheduler:54 - Job 12 finished: isEmpty at SpeedLayerUpdate.java:53, took 0.048293 s
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  SparkContext:54 - Starting job: isEmpty at SpeedLayerUpdate.java:53
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  DAGScheduler:54 - Got job 13 (isEmpty at SpeedLayerUpdate.java:53) with 3 output partitions
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  DAGScheduler:54 - Final stage: ResultStage 13 (isEmpty at SpeedLayerUpdate.java:53)
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  DAGScheduler:54 - Parents of final stage: List()
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  DAGScheduler:54 - Missing parents: List()
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  DAGScheduler:54 - Submitting ResultStage 13 (MapPartitionsRDD[15] at mapToPair at SpeedLayer.java:97), which has no missing parents
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  MemoryStore:54 - Block broadcast_13 stored as values in memory (estimated size 3.8 KB, free 912.3 MB)
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  MemoryStore:54 - Block broadcast_13_piece0 stored as bytes in memory (estimated size 2.4 KB, free 912.3 MB)
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  BlockManagerInfo:54 - Added broadcast_13_piece0 in memory on htm-psycho-401.zxz.su:42317 (size: 2.4 KB, free: 912.3 MB)
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  SparkContext:54 - Created broadcast 13 from broadcast at DAGScheduler.scala:1039
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  DAGScheduler:54 - Submitting 3 missing tasks from ResultStage 13 (MapPartitionsRDD[15] at mapToPair at SpeedLayer.java:97) (first 15 tasks are for partitions Vector(1, 2, 3))
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  YarnScheduler:54 - Adding task set 13.0 with 3 tasks
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  TaskSetManager:54 - Starting task 0.0 in stage 13.0 (TID 23, htm-psycho-401.zxz.su, executor 1, partition 1, PROCESS_LOCAL, 7746 bytes)
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  BlockManagerInfo:54 - Added broadcast_13_piece0 in memory on htm-psycho-401.zxz.su:45221 (size: 2.4 KB, free: 912.3 MB)
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  TaskSetManager:54 - Starting task 1.0 in stage 13.0 (TID 24, htm-psycho-401.zxz.su, executor 1, partition 2, PROCESS_LOCAL, 7746 bytes)
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  TaskSetManager:54 - Finished task 0.0 in stage 13.0 (TID 23) in 31 ms on htm-psycho-401.zxz.su (executor 1) (1/3)
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  TaskSetManager:54 - Starting task 2.0 in stage 13.0 (TID 25, htm-psycho-401.zxz.su, executor 1, partition 3, PROCESS_LOCAL, 7746 bytes)
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  TaskSetManager:54 - Finished task 1.0 in stage 13.0 (TID 24) in 13 ms on htm-psycho-401.zxz.su (executor 1) (2/3)
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  TaskSetManager:54 - Finished task 2.0 in stage 13.0 (TID 25) in 16 ms on htm-psycho-401.zxz.su (executor 1) (3/3)
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  YarnScheduler:54 - Removed TaskSet 13.0, whose tasks have all completed, from pool
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  DAGScheduler:54 - ResultStage 13 (isEmpty at SpeedLayerUpdate.java:53) finished in 0.072 s
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  DAGScheduler:54 - Job 13 finished: isEmpty at SpeedLayerUpdate.java:53, took 0.078273 s
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  JobScheduler:54 - Finished job streaming job 1536273300000 ms.0 from job set of time 1536273300000 ms
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  JobScheduler:54 - Starting job streaming job 1536273300000 ms.1 from job set of time 1536273300000 ms
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  UpdateOffsetsFn:60 - Updating offsets: {OryxInput,0=2576784, OryxInput,1=2579068, OryxInput,2=2577833, OryxInput,3=2579317}
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  ZooKeeper:438 - Initiating client connection, connectString=127.0.0.1:2181 sessionTimeout=30000 watcher=org.I0Itec.zkclient.ZkClient@18590535
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  ZkEventThread:65 - Starting ZkClient event thread.
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  ZkClient:936 - Waiting for keeper state SyncConnected
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  ClientCnxn:975 - Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  ClientCnxn:852 - Socket connection established to 127.0.0.1/127.0.0.1:2181, initiating session
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  ClientCnxn:1235 - Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x10160d98a7901f4, negotiated timeout = 30000
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  ZkClient:713 - zookeeper state changed (SyncConnected)
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  ZkEventThread:83 - Terminate ZkClient event thread.
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  ZooKeeper:684 - Session: 0x10160d98a7901f4 closed
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  ClientCnxn:512 - EventThread shut down
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  JobScheduler:54 - Finished job streaming job 1536273300000 ms.1 from job set of time 1536273300000 ms
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  JobScheduler:54 - Total delay: 0.162 s for time 1536273300000 ms (execution: 0.156 s)
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  MapPartitionsRDD:54 - Removing RDD 13 from persistence list
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  BlockManager:54 - Removing RDD 13
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  KafkaRDD:54 - Removing RDD 12 from persistence list
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  ReceivedBlockTracker:54 - Deleting batches:
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  InputInfoTracker:54 - remove old batch metadata: 1536272700000 ms
Sep 07 01:35:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:35:00 INFO  BlockManager:54 - Removing RDD 12
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  JobScheduler:54 - Starting job streaming job 1536273600000 ms.0 from job set of time 1536273600000 ms
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  JobScheduler:54 - Added jobs for time 1536273600000 ms
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  SparkContext:54 - Starting job: isEmpty at SpeedLayerUpdate.java:53
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  DAGScheduler:54 - Got job 14 (isEmpty at SpeedLayerUpdate.java:53) with 1 output partitions
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  DAGScheduler:54 - Final stage: ResultStage 14 (isEmpty at SpeedLayerUpdate.java:53)
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  DAGScheduler:54 - Parents of final stage: List()
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  DAGScheduler:54 - Missing parents: List()
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  DAGScheduler:54 - Submitting ResultStage 14 (MapPartitionsRDD[17] at mapToPair at SpeedLayer.java:97), which has no missing parents
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  MemoryStore:54 - Block broadcast_14 stored as values in memory (estimated size 3.8 KB, free 912.3 MB)
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  MemoryStore:54 - Block broadcast_14_piece0 stored as bytes in memory (estimated size 2.4 KB, free 912.3 MB)
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  BlockManagerInfo:54 - Added broadcast_14_piece0 in memory on htm-psycho-401.zxz.su:42317 (size: 2.4 KB, free: 912.3 MB)
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  SparkContext:54 - Created broadcast 14 from broadcast at DAGScheduler.scala:1039
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 14 (MapPartitionsRDD[17] at mapToPair at SpeedLayer.java:97) (first 15 tasks are for partitions Vector(0))
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  YarnScheduler:54 - Adding task set 14.0 with 1 tasks
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  TaskSetManager:54 - Starting task 0.0 in stage 14.0 (TID 26, htm-psycho-401.zxz.su, executor 1, partition 0, PROCESS_LOCAL, 7746 bytes)
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  BlockManagerInfo:54 - Added broadcast_14_piece0 in memory on htm-psycho-401.zxz.su:45221 (size: 2.4 KB, free: 912.3 MB)
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  TaskSetManager:54 - Finished task 0.0 in stage 14.0 (TID 26) in 55 ms on htm-psycho-401.zxz.su (executor 1) (1/1)
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  YarnScheduler:54 - Removed TaskSet 14.0, whose tasks have all completed, from pool
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  DAGScheduler:54 - ResultStage 14 (isEmpty at SpeedLayerUpdate.java:53) finished in 0.068 s
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  DAGScheduler:54 - Job 14 finished: isEmpty at SpeedLayerUpdate.java:53, took 0.071840 s
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  JobScheduler:54 - Finished job streaming job 1536273600000 ms.0 from job set of time 1536273600000 ms
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  JobScheduler:54 - Starting job streaming job 1536273600000 ms.1 from job set of time 1536273600000 ms
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  UpdateOffsetsFn:60 - Updating offsets: {OryxInput,0=2626356, OryxInput,1=2629184, OryxInput,2=2627875, OryxInput,3=2629587}
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  ZooKeeper:438 - Initiating client connection, connectString=127.0.0.1:2181 sessionTimeout=30000 watcher=org.I0Itec.zkclient.ZkClient@710d987a
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  ZkEventThread:65 - Starting ZkClient event thread.
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  ZkClient:936 - Waiting for keeper state SyncConnected
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  ClientCnxn:975 - Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  ClientCnxn:852 - Socket connection established to 127.0.0.1/127.0.0.1:2181, initiating session
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  ClientCnxn:1235 - Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x10160d98a7901f7, negotiated timeout = 30000
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  ZkClient:713 - zookeeper state changed (SyncConnected)
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  ZkEventThread:83 - Terminate ZkClient event thread.
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  ZooKeeper:684 - Session: 0x10160d98a7901f7 closed
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  ClientCnxn:512 - EventThread shut down
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  JobScheduler:54 - Finished job streaming job 1536273600000 ms.1 from job set of time 1536273600000 ms
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  JobScheduler:54 - Total delay: 0.118 s for time 1536273600000 ms (execution: 0.113 s)
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  MapPartitionsRDD:54 - Removing RDD 15 from persistence list
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  BlockManager:54 - Removing RDD 15
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  KafkaRDD:54 - Removing RDD 14 from persistence list
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  BlockManager:54 - Removing RDD 14
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  ReceivedBlockTracker:54 - Deleting batches:
Sep 07 01:40:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:40:00 INFO  InputInfoTracker:54 - remove old batch metadata: 1536273000000 ms
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  JobScheduler:54 - Added jobs for time 1536273900000 ms
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  JobScheduler:54 - Starting job streaming job 1536273900000 ms.0 from job set of time 1536273900000 ms
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  SparkContext:54 - Starting job: isEmpty at SpeedLayerUpdate.java:53
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  DAGScheduler:54 - Got job 15 (isEmpty at SpeedLayerUpdate.java:53) with 1 output partitions
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  DAGScheduler:54 - Final stage: ResultStage 15 (isEmpty at SpeedLayerUpdate.java:53)
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  DAGScheduler:54 - Parents of final stage: List()
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  DAGScheduler:54 - Missing parents: List()
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  DAGScheduler:54 - Submitting ResultStage 15 (MapPartitionsRDD[19] at mapToPair at SpeedLayer.java:97), which has no missing parents
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  MemoryStore:54 - Block broadcast_15 stored as values in memory (estimated size 3.8 KB, free 912.3 MB)
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  MemoryStore:54 - Block broadcast_15_piece0 stored as bytes in memory (estimated size 2.4 KB, free 912.3 MB)
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  BlockManagerInfo:54 - Added broadcast_15_piece0 in memory on htm-psycho-401.zxz.su:42317 (size: 2.4 KB, free: 912.3 MB)
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  SparkContext:54 - Created broadcast 15 from broadcast at DAGScheduler.scala:1039
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 15 (MapPartitionsRDD[19] at mapToPair at SpeedLayer.java:97) (first 15 tasks are for partitions Vector(0))
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  YarnScheduler:54 - Adding task set 15.0 with 1 tasks
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  TaskSetManager:54 - Starting task 0.0 in stage 15.0 (TID 27, htm-psycho-401.zxz.su, executor 1, partition 0, PROCESS_LOCAL, 7746 bytes)
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  BlockManagerInfo:54 - Added broadcast_15_piece0 in memory on htm-psycho-401.zxz.su:45221 (size: 2.4 KB, free: 912.3 MB)
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  TaskSetManager:54 - Finished task 0.0 in stage 15.0 (TID 27) in 27 ms on htm-psycho-401.zxz.su (executor 1) (1/1)
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  YarnScheduler:54 - Removed TaskSet 15.0, whose tasks have all completed, from pool
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  DAGScheduler:54 - ResultStage 15 (isEmpty at SpeedLayerUpdate.java:53) finished in 0.042 s
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  DAGScheduler:54 - Job 15 finished: isEmpty at SpeedLayerUpdate.java:53, took 0.046513 s
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  SparkContext:54 - Starting job: isEmpty at SpeedLayerUpdate.java:53
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  DAGScheduler:54 - Got job 16 (isEmpty at SpeedLayerUpdate.java:53) with 3 output partitions
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  DAGScheduler:54 - Final stage: ResultStage 16 (isEmpty at SpeedLayerUpdate.java:53)
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  DAGScheduler:54 - Parents of final stage: List()
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  DAGScheduler:54 - Missing parents: List()
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  DAGScheduler:54 - Submitting ResultStage 16 (MapPartitionsRDD[19] at mapToPair at SpeedLayer.java:97), which has no missing parents
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  MemoryStore:54 - Block broadcast_16 stored as values in memory (estimated size 3.8 KB, free 912.3 MB)
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  MemoryStore:54 - Block broadcast_16_piece0 stored as bytes in memory (estimated size 2.4 KB, free 912.3 MB)
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  BlockManagerInfo:54 - Added broadcast_16_piece0 in memory on htm-psycho-401.zxz.su:42317 (size: 2.4 KB, free: 912.3 MB)
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  SparkContext:54 - Created broadcast 16 from broadcast at DAGScheduler.scala:1039
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  DAGScheduler:54 - Submitting 3 missing tasks from ResultStage 16 (MapPartitionsRDD[19] at mapToPair at SpeedLayer.java:97) (first 15 tasks are for partitions Vector(1, 2, 3))
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  YarnScheduler:54 - Adding task set 16.0 with 3 tasks
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  TaskSetManager:54 - Starting task 0.0 in stage 16.0 (TID 28, htm-psycho-401.zxz.su, executor 1, partition 1, PROCESS_LOCAL, 7746 bytes)
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  BlockManagerInfo:54 - Added broadcast_16_piece0 in memory on htm-psycho-401.zxz.su:45221 (size: 2.4 KB, free: 912.3 MB)
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  TaskSetManager:54 - Starting task 1.0 in stage 16.0 (TID 29, htm-psycho-401.zxz.su, executor 1, partition 2, PROCESS_LOCAL, 7746 bytes)
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  TaskSetManager:54 - Finished task 0.0 in stage 16.0 (TID 28) in 26 ms on htm-psycho-401.zxz.su (executor 1) (1/3)
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  TaskSetManager:54 - Starting task 2.0 in stage 16.0 (TID 30, htm-psycho-401.zxz.su, executor 1, partition 3, PROCESS_LOCAL, 7746 bytes)
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  TaskSetManager:54 - Finished task 1.0 in stage 16.0 (TID 29) in 15 ms on htm-psycho-401.zxz.su (executor 1) (2/3)
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  TaskSetManager:54 - Finished task 2.0 in stage 16.0 (TID 30) in 17 ms on htm-psycho-401.zxz.su (executor 1) (3/3)
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  YarnScheduler:54 - Removed TaskSet 16.0, whose tasks have all completed, from pool
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  DAGScheduler:54 - ResultStage 16 (isEmpty at SpeedLayerUpdate.java:53) finished in 0.069 s
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  DAGScheduler:54 - Job 16 finished: isEmpty at SpeedLayerUpdate.java:53, took 0.073463 s
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  JobScheduler:54 - Finished job streaming job 1536273900000 ms.0 from job set of time 1536273900000 ms
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  JobScheduler:54 - Starting job streaming job 1536273900000 ms.1 from job set of time 1536273900000 ms
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  UpdateOffsetsFn:60 - Updating offsets: {OryxInput,0=2626356, OryxInput,1=2629184, OryxInput,2=2627875, OryxInput,3=2629587}
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  ZooKeeper:438 - Initiating client connection, connectString=127.0.0.1:2181 sessionTimeout=30000 watcher=org.I0Itec.zkclient.ZkClient@2796e67b
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  ZkEventThread:65 - Starting ZkClient event thread.
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  ZkClient:936 - Waiting for keeper state SyncConnected
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  ClientCnxn:975 - Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  ClientCnxn:852 - Socket connection established to 127.0.0.1/127.0.0.1:2181, initiating session
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  ClientCnxn:1235 - Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x10160d98a7901f8, negotiated timeout = 30000
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  ZkClient:713 - zookeeper state changed (SyncConnected)
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  ZkEventThread:83 - Terminate ZkClient event thread.
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  ZooKeeper:684 - Session: 0x10160d98a7901f8 closed
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  ClientCnxn:512 - EventThread shut down
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  JobScheduler:54 - Finished job streaming job 1536273900000 ms.1 from job set of time 1536273900000 ms
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  JobScheduler:54 - Total delay: 0.153 s for time 1536273900000 ms (execution: 0.148 s)
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  MapPartitionsRDD:54 - Removing RDD 17 from persistence list
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  KafkaRDD:54 - Removing RDD 16 from persistence list
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  BlockManager:54 - Removing RDD 17
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  BlockManager:54 - Removing RDD 16
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  ReceivedBlockTracker:54 - Deleting batches:
Sep 07 01:45:00 htm-psycho-401.zxz.su bash[9121]: 2018-09-07 01:45:00 INFO  InputInfoTracker:54 - remove old batch metadata: 1536273300000 ms

serving:

Sep 07 00:57:07 htm-psycho-401.zxz.su java[9123]: 2018-09-07 00:57:07,892 INFO  AppInfoParser:83 Kafka version : 0.11.0.1
Sep 07 00:57:07 htm-psycho-401.zxz.su java[9123]: 2018-09-07 00:57:07,892 INFO  AppInfoParser:84 Kafka commitId : c2a0d5f9b1f45bf5
Sep 07 00:57:08 htm-psycho-401.zxz.su java[9123]: 2018-09-07 00:57:08,075 INFO  AbstractCoordinator:607 Discovered coordinator localhost:9092 (id: 2147483647 rack: null) for group OryxGroup-ServingLayer-157da7e9-f0e9-468e-bb5e-a7293cd7e111.
Sep 07 00:57:08 htm-psycho-401.zxz.su java[9123]: 2018-09-07 00:57:08,078 INFO  ConsumerCoordinator:419 Revoking previously assigned partitions [] for group OryxGroup-ServingLayer-157da7e9-f0e9-468e-bb5e-a7293cd7e111
Sep 07 00:57:08 htm-psycho-401.zxz.su java[9123]: 2018-09-07 00:57:08,078 INFO  AbstractCoordinator:442 (Re-)joining group OryxGroup-ServingLayer-157da7e9-f0e9-468e-bb5e-a7293cd7e111
Sep 07 00:57:08 htm-psycho-401.zxz.su java[9123]: 2018-09-07 00:57:08,100 INFO  AbstractCoordinator:409 Successfully joined group OryxGroup-ServingLayer-157da7e9-f0e9-468e-bb5e-a7293cd7e111 with generation 1
Sep 07 00:57:08 htm-psycho-401.zxz.su java[9123]: 2018-09-07 00:57:08,101 INFO  ConsumerCoordinator:262 Setting newly assigned partitions [OryxUpdate-0] for group OryxGroup-ServingLayer-157da7e9-f0e9-468e-bb5e-a7293cd7e111
Sep 07 00:57:08 htm-psycho-401.zxz.su java[9123]: 2018-09-07 00:57:08,261 INFO  OryxApplication:65 Creating JAX-RS from endpoints in package(s) com.cloudera.oryx.app.serving,com.cloudera.oryx.app.serving.als,com.cloudera.oryx.lambda.serving
Sep 07 00:57:08 htm-psycho-401.zxz.su java[9123]: 2018-09-07 00:57:08,412 INFO  Reflections:229 Reflections took 126 ms to scan 1 urls, producing 17 keys and 106 values
Sep 07 00:57:08 htm-psycho-401.zxz.su java[9123]: 2018-09-07 00:57:08,511 INFO  Reflections:229 Reflections took 68 ms to scan 1 urls, producing 10 keys and 64 values
Sep 07 00:57:08 htm-psycho-401.zxz.su java[9123]: 2018-09-07 00:57:08,565 INFO  Reflections:229 Reflections took 50 ms to scan 1 urls, producing 13 keys and 14 values
Sep 07 00:57:09 htm-psycho-401.zxz.su java[9123]: Sep 07, 2018 12:57:09 AM org.apache.coyote.AbstractProtocol start
Sep 07 00:57:09 htm-psycho-401.zxz.su java[9123]: INFO: Starting ProtocolHandler ["http-nio2-8080"]
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: 2018-09-07 01:00:50,141 INFO  ProducerConfig:223 ProducerConfig values:
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: acks = 1
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: batch.size = 16384
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: bootstrap.servers = [127.0.0.1:9092]
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: buffer.memory = 33554432
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: client.id =
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: compression.type = gzip
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: connections.max.idle.ms = 540000
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: enable.idempotence = false
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: interceptor.classes = null
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: key.serializer = class org.apache.kafka.common.serialization.StringSerializer
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: linger.ms = 1000
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: max.block.ms = 60000
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: max.in.flight.requests.per.connection = 5
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: max.request.size = 67108864
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: metadata.max.age.ms = 300000
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: metric.reporters = []
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: metrics.num.samples = 2
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: metrics.recording.level = INFO
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: metrics.sample.window.ms = 30000
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: receive.buffer.bytes = 32768
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: reconnect.backoff.max.ms = 1000
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: reconnect.backoff.ms = 50
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: request.timeout.ms = 30000
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: retries = 0
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: retry.backoff.ms = 100
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: sasl.jaas.config = null
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: sasl.kerberos.kinit.cmd = /usr/bin/kinit
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: sasl.kerberos.min.time.before.relogin = 60000
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: sasl.kerberos.service.name = null
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: sasl.kerberos.ticket.renew.jitter = 0.05
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: sasl.kerberos.ticket.renew.window.factor = 0.8
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: sasl.mechanism = GSSAPI
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: security.protocol = PLAINTEXT
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: send.buffer.bytes = 131072
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: ssl.cipher.suites = null
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: ssl.endpoint.identification.algorithm = null
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: ssl.key.password = null
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: ssl.keymanager.algorithm = SunX509
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: ssl.keystore.location = null
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: ssl.keystore.password = null
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: ssl.keystore.type = JKS
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: ssl.protocol = TLS
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: ssl.provider = null
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: ssl.secure.random.implementation = null
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: ssl.trustmanager.algorithm = PKIX
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: ssl.truststore.location = null
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: ssl.truststore.password = null
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: ssl.truststore.type = JKS
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: transaction.timeout.ms = 60000
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: transactional.id = null
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: value.serializer = class org.apache.kafka.common.serialization.StringSerializer
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: 2018-09-07 01:00:50,156 INFO  AppInfoParser:83 Kafka version : 0.11.0.1
Sep 07 01:00:50 htm-psycho-401.zxz.su java[9123]: 2018-09-07 01:00:50,156 INFO  AppInfoParser:84 Kafka commitId : c2a0d5f9b1f45bf5

why serving layer still gives 503?

Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 INFO YarnSchedulerBackend$YarnDriverEndpoint:54 - Disabling executor 1. Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 INFO DAGScheduler:54 - Executor lost: 1 (epoch 15) Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 INFO BlockManagerMasterEndpoint:54 - Trying to remove executor 1 from BlockManagerMaster. Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 ERROR TransportClient:233 - Failed to send RPC 4632003068203477306 to /192.168.147.4:58358: java.nio.channels.ClosedChannelException Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: java.nio.channels.ClosedChannelException Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: at io.netty.channel.AbstractChannel$AbstractUnsafe.write(...)(Unknown Source) Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_7 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_22 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_31 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_119_4 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_61_0 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_119_8 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_61_12 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_8 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_1 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_66_11 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_109_13 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_5 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_119_9 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_20 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_67_2 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_62_8 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_67_8 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_119_7 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_61_2 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_2 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_119_11 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN YarnSchedulerBackend$YarnSchedulerEndpoint:87 - Attempted to get executor loss reason for executor id 1 at RPC address 192.168.147.4:58368, but got no response. Marking as slave lost. Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: java.io.IOException: Failed to send RPC 4632003068203477306 to /192.168.147.4:58358: java.nio.channels.ClosedChannelException Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: at org.apache.spark.network.client.TransportClient.lambda$sendRpc$2(TransportClient.java:237) Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507) Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:481) Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420) Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:122) Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: at io.netty.channel.AbstractChannel$AbstractUnsafe.safeSetFailure(AbstractChannel.java:987) Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: at io.netty.channel.AbstractChannel$AbstractUnsafe.write(AbstractChannel.java:869) Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: at io.netty.channel.DefaultChannelPipeline$HeadContext.write(DefaultChannelPipeline.java:1316) Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:738) Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:730) Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: at io.netty.channel.AbstractChannelHandlerContext.access$1900(AbstractChannelHandlerContext.java:38) Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.write(AbstractChannelHandlerContext.java:1081) Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: at io.netty.channel.AbstractChannelHandlerContext$WriteAndFlushTask.write(AbstractChannelHandlerContext.java:1128) Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.run(AbstractChannelHandlerContext.java:1070) Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403) Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463) Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138) Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: at java.lang.Thread.run(Thread.java:748) Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: Caused by: java.nio.channels.ClosedChannelException Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: at io.netty.channel.AbstractChannel$AbstractUnsafe.write(...)(Unknown Source) Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_67_0 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_61_11 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_62_9 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_109_2 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_67_3 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_61_4 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_119_10 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_61_3 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_10 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_15 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_30 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_62_11 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_67_1 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_26 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_19 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_0 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_109_7 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_2 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_9 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_12_28 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_66_1 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_31 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_109_14 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_66_13 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_62_14 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_61_8 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_61_7 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_61_10 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_66_9 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_61_1 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_109_3 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_61_14 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_66_6 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_13 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_119_6 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_5_26 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_6 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_4 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_3 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_109_12 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_12_30 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_66_12 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_66_0 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_109_15 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_17 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_5_21 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_12_27 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_66_8 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_109_6 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_14 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_67_6 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_6 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_25 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_5_25 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_67_15 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_119_14 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_13 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_66_4 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_11 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_29 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_24 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_0 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_26 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_62_3 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_5_24 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_23 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_5 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_18 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_61_6 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_61_15 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_11 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_7 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_62_1 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_8 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_61_5 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_22 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_119_5 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_14 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_30 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_109_5 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 ERROR YarnScheduler:70 - Lost executor 1 on htm-psycho-401.zxz.su: Slave lost Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_66_3 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_66_5 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_67_4 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_16 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_66_10 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_61_9 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_67_11 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_62_6 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_66_7 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_67_9 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_20 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_9 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_109_0 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_119_12 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_62_15 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_12 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_67_7 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_109_11 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_12_25 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_62_13 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_61_13 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_119_0 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_15 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_28 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_62_2 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_109_8 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_5_27 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_27 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_119_13 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_67_13 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_19 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_66_14 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_62_5 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_119_2 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_3 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_28 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_18 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_66_2 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_67_12 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_109_4 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_62_10 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_29 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_62_4 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_119_3 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_21 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_5_23 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_66_15 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_17 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_1 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_21 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_67_5 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_67_10 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_24 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_12 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_4 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_109_9 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_25 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_67_14 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_23 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_109_1 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_109_10 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_12_31 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_27 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_62_0 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN TaskSetManager:66 - Lost task 15.0 in stage 94.0 (TID 778, htm-psycho-401.zxz.su, executor 1): ExecutorLostFailure (executor 1 exited caused by one of the running tasks) Reason: Slave lost Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_12_29 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_62_7 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_58_10 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_52_16 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_119_1 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 WARN BlockManagerMasterEndpoint:66 - No more replicas available for rdd_62_12 ! Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 INFO BlockManagerMasterEndpoint:54 - Removing block manager BlockManagerId(1, htm-psycho-401.zxz.su, 41761, None) Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 INFO BlockManagerMaster:54 - Removed 1 successfully in removeExecutor Sep 07 01:58:16 htm-psycho-401.zxz.su bash[10427]: 2018-09-07 01:58:16 INFO DAGScheduler:54 - Shuffle files lost for executor: 1 (epoch 15)

OryxProject / oryx

KryoException: Buffer overflow. Available: 1, required: 4 #350