search

covolution / squick

Spark Quick start project
Apache License 2.0
0 stars 1 forks source link

Cassandra Connector doesn't work. #1

Open covolution opened 8 years ago

covolution commented 8 years ago

Using the Cassandra Connector 1.5.0-M3 it pulls in cassandra-driver-core 3.0.0-alpha4 which has this error.

CassandraConnector: Connected to Cassandra cluster: Test Cluster
Exception in thread "main" java.util.NoSuchElementException: key not found: 'text'
    at scala.collection.MapLike$class.default(MapLike.scala:228)
    at scala.collection.AbstractMap.default(Map.scala:58)
    at scala.collection.MapLike$class.apply(MapLike.scala:141)
    at scala.collection.AbstractMap.apply(Map.scala:58)
    at com.datastax.spark.connector.types.ColumnType$.fromDriverType(ColumnType.scala:81)
    at com.datastax.spark.connector.cql.ColumnDef$.apply(Schema.scala:117)

    at org.covolution.quick.App$.main(App.scala:33)

If we explicitly specify the last cassandra-driver-core as 3.0.0-beta1 then we get this error

15/12/21 11:12:23 INFO BlockManagerMaster: Registered BlockManager
Exception in thread "main" java.lang.NoSuchFieldError: DEFAULT_SSL_CIPHER_SUITES
    at com.datastax.spark.connector.cql.CassandraConnectorConf$CassandraSSLConf$.apply$default$6(CassandraConnectorConf.scala:49)
    at com.datastax.spark.connector.cql.CassandraConnectorConf$.<init>(CassandraConnectorConf.scala:180)
    at com.datastax.spark.connector.cql.CassandraConnectorConf$.<clinit>(CassandraConnectorConf.scala)
    at com.datastax.spark.connector.cql.CassandraConnector$.apply(CassandraConnector.scala:190)
    at org.covolution.quick.App$.main(App.scala:24)
covolution commented 8 years ago

Cassandra Connector 1.5.0-M3 has the follow transitive dependencies:

[INFO] +- com.datastax.spark:spark-cassandra-connector_2.10:jar:1.5.0-M3:compile
[INFO] |  +- org.apache.spark:spark-repl_2.10:jar:1.5.1:compile
[INFO] |  |  +- org.apache.spark:spark-bagel_2.10:jar:1.5.1:runtime
[INFO] |  |  +- org.apache.spark:spark-mllib_2.10:jar:1.5.1:runtime
[INFO] |  |  |  +- org.apache.spark:spark-graphx_2.10:jar:1.5.1:runtime
[INFO] |  |  |  |  +- com.github.fommil.netlib:core:jar:1.1.2:runtime
[INFO] |  |  |  |  \- net.sourceforge.f2j:arpack_combined_all:jar:0.1:runtime
[INFO] |  |  |  +- org.scalanlp:breeze_2.10:jar:0.11.2:runtime
[INFO] |  |  |  |  +- org.scalanlp:breeze-macros_2.10:jar:0.11.2:runtime
[INFO] |  |  |  |  +- net.sf.opencsv:opencsv:jar:2.3:runtime
[INFO] |  |  |  |  +- com.github.rwl:jtransforms:jar:2.4.0:runtime
[INFO] |  |  |  |  \- org.spire-math:spire_2.10:jar:0.7.4:runtime
[INFO] |  |  |  |     \- org.spire-math:spire-macros_2.10:jar:0.7.4:runtime
[INFO] |  |  |  \- org.jpmml:pmml-model:jar:1.1.15:runtime
[INFO] |  |  |     +- org.jpmml:pmml-agent:jar:1.1.15:runtime
[INFO] |  |  |     +- org.jpmml:pmml-schema:jar:1.1.15:runtime
[INFO] |  |  |     \- com.sun.xml.bind:jaxb-impl:jar:2.2.7:runtime
[INFO] |  |  |        \- com.sun.xml.bind:jaxb-core:jar:2.2.7:runtime
[INFO] |  |  |           \- javax.xml.bind:jaxb-api:jar:2.2.7:runtime
[INFO] |  |  \- org.scala-lang:jline:jar:2.10.4:compile
[INFO] |  |     \- org.fusesource.jansi:jansi:jar:1.4:compile
[INFO] |  +- com.google.guava:guava:jar:16.0.1:compile
[INFO] |  +- org.joda:joda-convert:jar:1.2:compile
[INFO] |  +- joda-time:joda-time:jar:2.3:compile
[INFO] |  +- com.twitter:jsr166e:jar:1.1.0:compile
[INFO] |  \- org.scala-lang:scala-reflect:jar:2.10.5:compile
[INFO] |  +- org.apache.cassandra:cassandra-clientutil:jar:2.2.2:compile
[INFO] |  +- com.datastax.cassandra:cassandra-driver-core:jar:3.0.0-alpha4:compile
[INFO] |  |  +- io.netty:netty-handler:jar:4.0.27.Final:compile
[INFO] |  |  |  +- io.netty:netty-buffer:jar:4.0.27.Final:compile
[INFO] |  |  |  |  \- io.netty:netty-common:jar:4.0.27.Final:compile
[INFO] |  |  |  +- io.netty:netty-transport:jar:4.0.27.Final:compile
[INFO] |  |  |  \- io.netty:netty-codec:jar:4.0.27.Final:compile
[INFO] |  |  \- com.codahale.metrics:metrics-core:jar:3.0.2:compile

Cassandra Connector 1.5.0-M3 with an explicit cassandra-driver-core 3.0.0-beta1, has the follow transitive dependencies:

[INFO] +- com.datastax.spark:spark-cassandra-connector_2.10:jar:1.5.0-M3:compile
[INFO] |  +- org.apache.spark:spark-repl_2.10:jar:1.5.1:compile
[INFO] |  |  +- org.apache.spark:spark-bagel_2.10:jar:1.5.1:runtime
[INFO] |  |  +- org.apache.spark:spark-mllib_2.10:jar:1.5.1:runtime
[INFO] |  |  |  +- org.apache.spark:spark-graphx_2.10:jar:1.5.1:runtime
[INFO] |  |  |  |  +- com.github.fommil.netlib:core:jar:1.1.2:runtime
[INFO] |  |  |  |  \- net.sourceforge.f2j:arpack_combined_all:jar:0.1:runtime
[INFO] |  |  |  +- org.scalanlp:breeze_2.10:jar:0.11.2:runtime
[INFO] |  |  |  |  +- org.scalanlp:breeze-macros_2.10:jar:0.11.2:runtime
[INFO] |  |  |  |  +- net.sf.opencsv:opencsv:jar:2.3:runtime
[INFO] |  |  |  |  +- com.github.rwl:jtransforms:jar:2.4.0:runtime
[INFO] |  |  |  |  \- org.spire-math:spire_2.10:jar:0.7.4:runtime
[INFO] |  |  |  |     \- org.spire-math:spire-macros_2.10:jar:0.7.4:runtime
[INFO] |  |  |  \- org.jpmml:pmml-model:jar:1.1.15:runtime
[INFO] |  |  |     +- org.jpmml:pmml-agent:jar:1.1.15:runtime
[INFO] |  |  |     +- org.jpmml:pmml-schema:jar:1.1.15:runtime
[INFO] |  |  |     \- com.sun.xml.bind:jaxb-impl:jar:2.2.7:runtime
[INFO] |  |  |        \- com.sun.xml.bind:jaxb-core:jar:2.2.7:runtime
[INFO] |  |  |           \- javax.xml.bind:jaxb-api:jar:2.2.7:runtime
[INFO] |  |  \- org.scala-lang:jline:jar:2.10.4:compile
[INFO] |  |     \- org.fusesource.jansi:jansi:jar:1.4:compile
[INFO] |  +- com.google.guava:guava:jar:16.0.1:compile
[INFO] |  +- org.joda:joda-convert:jar:1.2:compile
[INFO] |  +- joda-time:joda-time:jar:2.3:compile
[INFO] |  +- com.twitter:jsr166e:jar:1.1.0:compile
[INFO] |  \- org.scala-lang:scala-reflect:jar:2.10.5:compile
[INFO] |  +- org.apache.cassandra:cassandra-clientutil:jar:2.2.2:compile
[INFO] +- com.datastax.cassandra:cassandra-driver-core:jar:3.0.0-beta1:compile
[INFO] |  \- io.netty:netty-handler:jar:4.0.33.Final:compile
[INFO] |     +- io.netty:netty-buffer:jar:4.0.33.Final:compile
[INFO] |     |  \- io.netty:netty-common:jar:4.0.33.Final:compile
[INFO] |     +- io.netty:netty-transport:jar:4.0.33.Final:compile
[INFO] |     \- io.netty:netty-codec:jar:4.0.33.Final:compile
covolution commented 8 years ago

Out of interest, the Cassandra Connector 1.5.0-M2 pull in far less transitive dependencies (because it doesn't depend on spark-repl): ´´´ [INFO] +- com.datastax.spark:spark-cassandra-connector_2.10:jar:1.5.0-M2:compile [INFO] | +- org.apache.cassandra:cassandra-clientutil:jar:2.1.5:compile [INFO] | +- com.datastax.cassandra:cassandra-driver-core:jar:2.2.0-rc3:compile [INFO] | | +- io.netty:netty-handler:jar:4.0.27.Final:compile [INFO] | | | +- io.netty:netty-buffer:jar:4.0.27.Final:compile [INFO] | | | | - io.netty:netty-common:jar:4.0.27.Final:compile [INFO] | | | +- io.netty:netty-transport:jar:4.0.27.Final:compile [INFO] | | | - io.netty:netty-codec:jar:4.0.27.Final:compile [INFO] | | - com.codahale.metrics:metrics-core:jar:3.0.2:compile [INFO] | +- com.google.guava:guava:jar:16.0.1:compile [INFO] | +- org.joda:joda-convert:jar:1.2:compile [INFO] | +- joda-time:joda-time:jar:2.3:compile [INFO] | +- com.twitter:jsr166e:jar:1.1.0:compile [INFO] | - org.scala-lang:scala-reflect:jar:2.10.5:compile ´´´

covolution commented 8 years ago

With Cassandra Connector 1.5.0-M2, it uses cassandra-driver-core:jar:2.2.0-rc3 and gives this error: java.io.IOException: Failed to open native connection to Cassandra at {172.17.0.2}:9042