LukeTillman
commented
7 years ago Thanks for reporting this. I've gone ahead and pushed up a new release for 5.1.4, as well as 5.1.5, both of which should have the fix.
Closed norim closed 7 years ago
LukeTillman
commented
7 years ago Thanks for reporting this. I've gone ahead and pushed up a new release for 5.1.4, as well as 5.1.5, both of which should have the fix.
Hi Luke,
just came across an issue with the directories for spark. The service just stops because of missing directories for Spark. See below the exception message. I were able to fix it if I add /var/lib/spark/worker /var/lib/spark/rdd to the dockerfile where the directories are created. Thanks, Negib
Version: luketillman/datastax-enterprise:5.1.4
ERROR [main] 2017-10-21 15:25:22,092 DseDaemon.java:499 - Unable to start DSE server. com.datastax.bdp.plugin.PluginManager$PluginActivationException: Unable to activate plugin com.datastax.bdp.plugin.SparkPlugin at com.datastax.bdp.plugin.PluginManager.activateDirect(PluginManager.java:422) ~[dse-core-5.1.4.jar:5.1.4] at com.datastax.bdp.plugin.PluginManager.activate(PluginManager.java:208) ~[dse-core-5.1.4.jar:5.1.4] at com.datastax.bdp.plugin.PluginManager.preStart(PluginManager.java:104) ~[dse-core-5.1.4.jar:5.1.4] at com.datastax.bdp.server.DseDaemon.preStart(DseDaemon.java:522) ~[dse-core-5.1.4.jar:5.1.4] at com.datastax.bdp.server.DseDaemon.start(DseDaemon.java:492) ~[dse-core-5.1.4.jar:5.1.4] at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:614) [cassandra-all-3.11.0.1900.jar:3.11.0.1900] at com.datastax.bdp.DseModule.main(DseModule.java:93) [dse-core-5.1.4.jar:5.1.4] Caused by: com.datastax.bdp.plugin.SparkPlugin$$anon$1: Spark Plugin requires the Spark directories to be created with rwx permissions for dse. The following problems have been found: /var/lib/spark/worker is missing, /var/lib/spark/rdd is missing. Please fix these problems and start DSE again Unable to start DSE server: Unable to activate plugin com.datastax.bdp.plugin.SparkPlugin com.datastax.bdp.plugin.PluginManager$PluginActivationException: Unable to activate plugin com.datastax.bdp.plugin.SparkPlugin at com.datastax.bdp.plugin.PluginManager.activateDirect(PluginManager.java:422) at com.datastax.bdp.plugin.PluginManager.activate(PluginManager.java:208) at com.datastax.bdp.plugin.PluginManager.preStart(PluginManager.java:104) at com.datastax.bdp.server.DseDaemon.preStart(DseDaemon.java:522) at com.datastax.bdp.server.DseDaemon.start(DseDaemon.java:492) at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:614) at com.datastax.bdp.DseModule.main(DseModule.java:93) Caused by: com.datastax.bdp.plugin.SparkPlugin$$anon$1: Spark Plugin requires the Spark directories to be created with rwx permissions for dse. The following problems have been found: /var/lib/spark/worker is missing, /var/lib/spark/rdd is missing. Please fix these problems and start DSE again Exception (java.lang.RuntimeException) encountered during startup: com.datastax.bdp.plugin.PluginManager$PluginActivationException: Unable to activate plugin com.datastax.bdp.plugin.SparkPlugin java.lang.RuntimeException: com.datastax.bdp.plugin.PluginManager$PluginActivationException: Unable to activate plugin com.datastax.bdp.plugin.SparkPlugin at com.datastax.bdp.server.DseDaemon.start(DseDaemon.java:502) at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:614) at com.datastax.bdp.DseModule.main(DseModule.java:93) Caused by: com.datastax.bdp.plugin.PluginManager$PluginActivationException: Unable to activate plugin com.datastax.bdp.plugin.SparkPlugin at com.datastax.bdp.plugin.PluginManager.activateDirect(PluginManager.java:422) at com.datastax.bdp.plugin.PluginManager.activate(PluginManager.java:208) at com.datastax.bdp.plugin.PluginManager.preStart(PluginManager.java:104) at com.datastax.bdp.server.DseDaemon.preStart(DseDaemon.java:522) at com.datastax.bdp.server.DseDaemon.start(DseDaemon.java:492) ... 2 more Caused by: com.datastax.bdp.plugin.SparkPlugin$$anon$1: Spark Plugin requires the Spark directories to be created with rwx permissions for dse. The following problems have been found: /var/lib/spark/worker is missing, /var/lib/spark/rdd is missing. Please fix these problems and start DSE again ERROR [main] 2017-10-21 15:25:22,095 CassandraDaemon.java:705 - Exception encountered during startup java.lang.RuntimeException: com.datastax.bdp.plugin.PluginManager$PluginActivationException: Unable to activate plugin com.datastax.bdp.plugin.SparkPlugin at com.datastax.bdp.server.DseDaemon.start(DseDaemon.java:502) ~[dse-core-5.1.4.jar:5.1.4] at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:614) ~[cassandra-all-3.11.0.1900.jar:3.11.0.1900] at com.datastax.bdp.DseModule.main(DseModule.java:93) [dse-core-5.1.4.jar:5.1.4] Caused by: com.datastax.bdp.plugin.PluginManager$PluginActivationException: Unable to activate plugin com.datastax.bdp.plugin.SparkPlugin at com.datastax.bdp.plugin.PluginManager.activateDirect(PluginManager.java:422) ~[dse-core-5.1.4.jar:5.1.4] at com.datastax.bdp.plugin.PluginManager.activate(PluginManager.java:208) ~[dse-core-5.1.4.jar:5.1.4] at com.datastax.bdp.plugin.PluginManager.preStart(PluginManager.java:104) ~[dse-core-5.1.4.jar:5.1.4] at com.datastax.bdp.server.DseDaemon.preStart(DseDaemon.java:522) ~[dse-core-5.1.4.jar:5.1.4] at com.datastax.bdp.server.DseDaemon.start(DseDaemon.java:492) ~[dse-core-5.1.4.jar:5.1.4] ... 2 common frames omitted Caused by: com.datastax.bdp.plugin.SparkPlugin$$anon$1: Spark Plugin requires the Spark directories to be created with rwx permissions for dse. The following problems have been found: /var/lib/spark/worker is missing, /var/lib/spark/rdd is missing. Please fix these problems and start DSE again INFO [StorageServiceShutdownHook] 2017-10-21 15:25:22,118 DseDaemon.java:892 - DSE shutting down... INFO [StorageServiceShutdownHook] 2017-10-21 15:25:22,120 PluginManager.java:128 - All plugins predeactivated. INFO [StorageServiceShutdownHook] 2017-10-21 15:25:26,549 MessageServer.java:129 - internode-messaging message server finished shutting down. INFO [StorageServiceShutdownHook] 2017-10-21 15:25:30,578 MessageServer.java:129 - internode-messaging message server finished shutting down. INFO [StorageServiceShutdownHook] 2017-10-21 15:25:30,579 HintsService.java:220 - Paused hints dispatch INFO [StorageServiceShutdownHook] 2017-10-21 15:25:30,585 Gossiper.java:1541 - Announcing shutdown INFO [StorageServiceShutdownHook] 2017-10-21 15:25:30,588 StorageService.java:2318 - Node /10.11.12.100 state jump to shutdown INFO [StorageServiceShutdownHook] 2017-10-21 15:25:32,589 MessagingService.java:984 - Waiting for messaging service to quiesce INFO [ACCEPT-/10.11.12.100] 2017-10-21 15:25:32,591 MessagingService.java:1338 - MessagingService has terminated the accept() thread INFO [StorageServiceShutdownHook] 2017-10-21 15:25:33,218 HintsService.java:220 - Paused hints dispatch INFO [StorageServiceShutdownHook] 2017-10-21 15:25:33,273 PluginManager.java:139 - All plugins are stopped. INFO [StorageServiceShutdownHook] 2017-10-21 15:25:33,274 DseDaemon.java:971 - DSE shutdown complete.