search

IBMDataScience / DSx-Desktop

IBM Data Science Experience Desktop was built for those who want to download and play locally. Analyze, learn, and build with the tools you love, right on your desktop.
33 stars 18 forks source link

Deprecated Spark functions in Spark for R.ipynb #9

Closed cabarendt closed 6 years ago

cabarendt commented 7 years ago

First two Spark APIs in this Tutorial Notebook generated Deprecated Warning messages:

(1) library(SparkR) sc <- sparkR.session() sqlContext <- sparkRSQL.init(sc)

Warning message: “'sparkRSQL.init' is deprecated. Use 'sparkR.session' instead. See help("Deprecated")”

(2) sdf <- createDataFrame(sqlContext, mtcars) printSchema(sdf)

Warning message: “'createDataFrame(sqlContext...)' is deprecated. Use 'createDataFrame(data, schema = NULL)' instead. See help("Deprecated")”

sahilshah2211 commented 6 years ago

@cabarendt Accoding to https://stackoverflow.com/questions/46434647/how-to-resolve-sqlsqlcontext-is-deprecated-warning-in-sparkr, since Spark 2.0 sql and the number of other functions (like createDataFrame) dont require SQLContext instance. Internally this will use getSparkSession to retrieve a session object.

jerry1100 commented 6 years ago

Closing due to lack of activity. Reopen if needed.