In some instances of Spark, the SparkContext, "sc", is already instantiated. Thus when the example is run, an error occurs :
'Cannot run multiple SparkContexts at once; existing SparkContext(app=PySparkShell, master=spark://ys1-spark-dal09-env4-0019:7082) created by <module> at /usr/local/src/bluemix_jupyter_bundle.v14/notebook/lib/python2.7/site-packages/IPython/utils/py3compat.py:286
The following changes to example.py fix this issue:
# Copyright (c) Jupyter Development Team.
# Distributed under the terms of the Modified BSD License.
import pyspark
if not globals().has_key('sc'):
sc = pyspark.SparkContext()
rdd = sc.parallelize(range(1000))
sample = rdd.takeSample(False, 5)
print(sample)
In some instances of Spark, the SparkContext, "sc", is already instantiated. Thus when the example is run, an error occurs :
The following changes to
example.py
fix this issue:Can that change be made to
example.py
?