17/09/26 20:27:51 INFO DStreamGraph: Clearing checkpoint data for time 1506457670000 ms
17/09/26 20:27:51 INFO DStreamCheckpointData: Deleted checkpoint file 'file:/home/ubuntu/spark-2.1.1-bin-hadoop2.7/bin/checkpoint/8d0a4b24-1d51-4723-a187-2faaff8039ec/rdd-240' for time 1506457650000 ms
17/09/26 20:27:51 INFO DStreamGraph: Cleared checkpoint data for time 1506457670000 ms
17/09/26 20:27:51 INFO ReceivedBlockTracker: Deleting batches:
17/09/26 20:27:51 WARN ReceivedBlockTracker: Exception thrown while writing record: BatchCleanupEvent(ArrayBuffer()) to the WriteAheadLog.
java.lang.IllegalStateException: close() was called on BatchedWriteAheadLog before write request with time 1506457671080 could be fulfilled.
at org.apache.spark.streaming.util.BatchedWriteAheadLog.write(BatchedWriteAheadLog.scala:86)
at org.apache.spark.streaming.scheduler.ReceivedBlockTracker.writeToLog(ReceivedBlockTracker.scala:234)
at org.apache.spark.streaming.scheduler.ReceivedBlockTracker.cleanupOldBatches(ReceivedBlockTracker.scala:171)
at org.apache.spark.streaming.scheduler.ReceiverTracker.cleanupOldBlocksAndBatches(ReceiverTracker.scala:233)
at org.apache.spark.streaming.scheduler.JobGenerator.clearCheckpointData(JobGenerator.scala:287)
at org.apache.spark.streaming.scheduler.JobGenerator.org$apache$spark$streaming$scheduler$JobGenerator$$processEvent(JobGenerator.scala:187)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:89)
at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:88)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
17/09/26 20:27:51 WARN ReceivedBlockTracker: Failed to acknowledge batch clean up in the Write Ahead Log.
17/09/26 20:27:51 INFO InputInfoTracker: remove old batch metadata: 1506457640000 ms
17/09/26 20:27:51 INFO JobGenerator: Waited for jobs to be processed and checkpoints to be written
17/09/26 20:27:51 INFO CheckpointWriter: CheckpointWriter executor terminated? true, waited for 0 ms.
17/09/26 20:27:51 INFO JobGenerator: Stopped JobGenerator
17/09/26 20:27:51 INFO JobScheduler: Stopped JobScheduler
17/09/26 20:27:51 INFO StreamingContext: StreamingContext stopped successfully
17/09/26 20:27:51 INFO SparkUI: Stopped Spark web UI at http://0.0.0.0:4040
17/09/26 20:27:51 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
17/09/26 20:27:51 INFO MemoryStore: MemoryStore cleared
17/09/26 20:27:51 INFO BlockManager: BlockManager stopped
17/09/26 20:27:51 INFO BlockManagerMaster: BlockManagerMaster stopped
17/09/26 20:27:51 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
17/09/26 20:27:51 INFO SparkContext: Successfully stopped SparkContext
17/09/26 20:27:51 INFO SparkContext: SparkContext already stopped.
Traceback (most recent call last):
File "/home/ubuntu/spark-2.1.1-bin-hadoop2.7/bin/Twitter-Sentiment-Analysis-Using-Spark-Streaming-And-Kafka-master/twitterStream.py", line 109, in
main()
File "/home/ubuntu/spark-2.1.1-bin-hadoop2.7/bin/Twitter-Sentiment-Analysis-Using-Spark-Streaming-And-Kafka-master/twitterStream.py", line 19, in main
make_plot(counts)
File "/home/ubuntu/spark-2.1.1-bin-hadoop2.7/bin/Twitter-Sentiment-Analysis-Using-Spark-Streaming-And-Kafka-master/twitterStream.py", line 31, in make_plot
positiveTuple = val[0]
IndexError: list index out of range
17/09/26 20:27:52 INFO ShutdownHookManager: Shutdown hook called
17/09/26 20:27:52 INFO ShutdownHookManager: Deleting directory /tmp/spark-6ee2dd3e-ecad-40ad-8db5-fffda52aca68
17/09/26 20:27:52 INFO ShutdownHookManager: Deleting directory /tmp/spark-6ee2dd3e-ecad-40ad-8db5-fffda52aca68/pyspark-196b74d1-cce4-4a5f-a468-c72410498d61
17/09/26 20:27:51 INFO DStreamGraph: Clearing checkpoint data for time 1506457670000 ms 17/09/26 20:27:51 INFO DStreamCheckpointData: Deleted checkpoint file 'file:/home/ubuntu/spark-2.1.1-bin-hadoop2.7/bin/checkpoint/8d0a4b24-1d51-4723-a187-2faaff8039ec/rdd-240' for time 1506457650000 ms 17/09/26 20:27:51 INFO DStreamGraph: Cleared checkpoint data for time 1506457670000 ms 17/09/26 20:27:51 INFO ReceivedBlockTracker: Deleting batches: 17/09/26 20:27:51 WARN ReceivedBlockTracker: Exception thrown while writing record: BatchCleanupEvent(ArrayBuffer()) to the WriteAheadLog. java.lang.IllegalStateException: close() was called on BatchedWriteAheadLog before write request with time 1506457671080 could be fulfilled. at org.apache.spark.streaming.util.BatchedWriteAheadLog.write(BatchedWriteAheadLog.scala:86) at org.apache.spark.streaming.scheduler.ReceivedBlockTracker.writeToLog(ReceivedBlockTracker.scala:234) at org.apache.spark.streaming.scheduler.ReceivedBlockTracker.cleanupOldBatches(ReceivedBlockTracker.scala:171) at org.apache.spark.streaming.scheduler.ReceiverTracker.cleanupOldBlocksAndBatches(ReceiverTracker.scala:233) at org.apache.spark.streaming.scheduler.JobGenerator.clearCheckpointData(JobGenerator.scala:287) at org.apache.spark.streaming.scheduler.JobGenerator.org$apache$spark$streaming$scheduler$JobGenerator$$processEvent(JobGenerator.scala:187) at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:89) at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:88) at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48) 17/09/26 20:27:51 WARN ReceivedBlockTracker: Failed to acknowledge batch clean up in the Write Ahead Log. 17/09/26 20:27:51 INFO InputInfoTracker: remove old batch metadata: 1506457640000 ms 17/09/26 20:27:51 INFO JobGenerator: Waited for jobs to be processed and checkpoints to be written 17/09/26 20:27:51 INFO CheckpointWriter: CheckpointWriter executor terminated? true, waited for 0 ms. 17/09/26 20:27:51 INFO JobGenerator: Stopped JobGenerator 17/09/26 20:27:51 INFO JobScheduler: Stopped JobScheduler 17/09/26 20:27:51 INFO StreamingContext: StreamingContext stopped successfully 17/09/26 20:27:51 INFO SparkUI: Stopped Spark web UI at http://0.0.0.0:4040 17/09/26 20:27:51 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped! 17/09/26 20:27:51 INFO MemoryStore: MemoryStore cleared 17/09/26 20:27:51 INFO BlockManager: BlockManager stopped 17/09/26 20:27:51 INFO BlockManagerMaster: BlockManagerMaster stopped 17/09/26 20:27:51 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped! 17/09/26 20:27:51 INFO SparkContext: Successfully stopped SparkContext 17/09/26 20:27:51 INFO SparkContext: SparkContext already stopped. Traceback (most recent call last): File "/home/ubuntu/spark-2.1.1-bin-hadoop2.7/bin/Twitter-Sentiment-Analysis-Using-Spark-Streaming-And-Kafka-master/twitterStream.py", line 109, in
main()
File "/home/ubuntu/spark-2.1.1-bin-hadoop2.7/bin/Twitter-Sentiment-Analysis-Using-Spark-Streaming-And-Kafka-master/twitterStream.py", line 19, in main
make_plot(counts)
File "/home/ubuntu/spark-2.1.1-bin-hadoop2.7/bin/Twitter-Sentiment-Analysis-Using-Spark-Streaming-And-Kafka-master/twitterStream.py", line 31, in make_plot
positiveTuple = val[0]
IndexError: list index out of range
17/09/26 20:27:52 INFO ShutdownHookManager: Shutdown hook called
17/09/26 20:27:52 INFO ShutdownHookManager: Deleting directory /tmp/spark-6ee2dd3e-ecad-40ad-8db5-fffda52aca68
17/09/26 20:27:52 INFO ShutdownHookManager: Deleting directory /tmp/spark-6ee2dd3e-ecad-40ad-8db5-fffda52aca68/pyspark-196b74d1-cce4-4a5f-a468-c72410498d61