saveToMemSQLApp.scala question

memsql / singlestore-spark-connector

A connector for SingleStore and Spark

Apache License 2.0

160 stars 54 forks source link

saveToMemSQLApp.scala question #7

Closed kimchitsigai closed 8 years ago

kimchitsigai commented 8 years ago

Hi,

I'm trying to run the example code in WriteToMemSQLApp.scala. I'm running it through spark-shell, as a yarn-client (my Spark version is 1.3.1).

When I run : val rdd = sc.parallelize(values) rdd.saveToMemSQL(dbName, outputTableName, host, port, user, password) I get an error because saveToMemsql is not a method of rdd.

It seems to me that sc is a org.apache.spark.SparkContext object and that sc.parallelize() returns an org.apache.spark.rdd.RDD object which has no saveToMemsql method.

Where am I wrong ?

Thanks a lot for any hint, Kimchitsigai

choochootrain commented 8 years ago

some things to check:

are the correct dependencies included when you start spark-shell? you will need to specify memsqlrdd (the name for 1.0.0) or memsql-connector (1.1.0 and above) in your --jars option
is your rdd of type RDD[Row]? saveToMemSQL is an implicit method that is only available for that type.

choochootrain commented 8 years ago

closing for now - this can be re-opened if you're still seeing this issue