Google cloud dataproc not recognizing 'sqlContext' command

spotify / spark-bigquery

Google BigQuery support for Spark, SQL, and DataFrames

Apache License 2.0

155 stars 52 forks source link

Google cloud dataproc not recognizing 'sqlContext' command #58

Open Indra2009 opened 6 years ago

Indra2009 commented 6 years ago

I am a newbie in Spark & Scala. I ran this package in local SBT and everything worked fine. But now I am trying to do the connection in Google cloud dataproc and executed the command:

spark-shell --packages com.spotify:spark-bigquery_2.10:0.2.0

But after that when I am trying to run the following command at scala prompt: val table = sqlContext.bigQueryTable("bigquery-public-data:samples.shakespeare") It can't recognize 'sqlContext' command. Am I missing something?

vatshy commented 6 years ago

Did you import types? import com.spotify.spark.bigquery._

Indra2009 commented 6 years ago

Yes after spark-shell command I imported the package. And after that I am trying to run the sqlcontext command. but it can't recognize it.

randoFelipe commented 5 years ago

any help with this?