Stratio Sandbox's stratio-deep-shell hangs running "first steps" tutorial

[heskech]

Sandbox version 0.91

[root@sandbox bin]# ./stratio-deep-shell SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/opt/sds/spark/lib/spark-assembly-1.0.0-hadoop1.0.4.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/opt/sds/spark/lib/spark-examples-1.0.0-hadoop1.0.4.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] Welcome to

---

/ / /__ / /(_) / _\ \/ / / _ `/ _/ / \ / // / -) -) \ //// ,_//// /__/// ./ /_/ Powered by Spark v1.0.0

Using Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.055) Type in expressions to have them evaluated. Type :help for more information. 23:38:25,730 INFO [spark-akka.actor.default-dispatcher-5] Slf4jLogger:80 - Slf4jLogger started 23:38:25,825 INFO [spark-akka.actor.default-dispatcher-5] Remoting:74 - Starting remoting 23:38:26,167 INFO [spark-akka.actor.default-dispatcher-5] Remoting:74 - Remoting started; listening on addresses :[akka.tcp://spark@sandbox:33621] 23:38:26,171 INFO [spark-akka.actor.default-dispatcher-3] Remoting:74 - Remoting now listens on addresses: [akka.tcp://spark@sandbox:33621] Spark context available as sc. Loading /opt/sds/spark/bin/stratio-deep-init.scala... import com.stratio.deep.annotations.DeepEntity import com.stratio.deep.annotations.DeepField import com.stratio.deep.entity.IDeepType import org.apache.cassandra.db.marshal.Int32Type import org.apache.cassandra.db.marshal.LongType import com.stratio.deep.config.{DeepJobConfigFactory=>Cfg, } import com.stratio.deep.entity. import com.stratio.deep.context. import com.stratio.deep.rdd. import com.stratio.deep.rdd.mongodb. import com.stratio.deep.testentity._ deepContext: com.stratio.deep.context.DeepSparkContext = com.stratio.deep.context.DeepSparkContext@227330c

scala> val config : ICassandraDeepJobConfig[Cells] = Cfg.create().host("localhost").rpcPort(9160).keyspace("crawler").table("Page").initialize config: com.stratio.deep.config.ICassandraDeepJobConfig[com.stratio.deep.entity.Cells] = com.stratio.deep.config.CellDeepJobConfig@6f16befb

scala> val rdd: CassandraRDD[Cells] = deepContext.cassandraGenericRDD(config) rdd: com.stratio.deep.rdd.CassandraRDD[com.stratio.deep.entity.Cells] = CassandraCellRDD[0] at RDD at CassandraRDD.java:173

scala> val containsAbcRDD = rdd filter {c :Cells => c.getCellByName("domainName").getCellValue.asInstanceOf[String].contains("abc.es") } containsAbcRDD: org.apache.spark.rdd.RDD[com.stratio.deep.entity.Cells] = FilteredRDD[1] at filter at :41

scala> containsAbcRDD.count

Hanging here

[heskech]

The Spark Master is reporting spark-shell as a running application with 0 cores which suggests it's hanging due to it having no resources.

[heskech]

This issue was caused as a side effect of the workaround I proposed to address issue #10.