search

jupyter-incubator / sparkmagic

Jupyter magics and kernels for working with remote Spark clusters
Other
1.33k stars 446 forks source link

Create hiveContext only -- sqlContext is not needed #166

Closed aggFTW closed 8 years ago

aggFTW commented 8 years ago

"In addition to the basic SQLContext, you can also create a HiveContext, which provides a superset of the functionality provided by the basic SQLContext. Additional features include the ability to write queries using the more complete HiveQL parser, access to Hive UDFs, and the ability to read data from Hive tables. To use a HiveContext, you do not need to have an existing Hive setup, and all of the data sources available to a SQLContext are still available. HiveContext is only packaged separately to avoid including all of Hive’s dependencies in the default Spark build. If these dependencies are not a problem for your application then using HiveContext is recommended for the 1.3 release of Spark. "

msftristew commented 8 years ago

Should be a pretty easy change IMO, we just need to:

  1. Remove all references to SQLContext, including the %%sql magic
  2. For uniformity with the Spark shell, we should probably also rename hiveContext to sqlContext (or sqlc?) and rename %%hive back to %%sql