search

jupyter-incubator / sparkmagic

Jupyter magics and kernels for working with remote Spark clusters
Other
1.33k stars 447 forks source link

'LivySession' object has no attribute 'sql_context_variable_name' #472

Open evshieh-msft opened 6 years ago

evshieh-msft commented 6 years ago

spark_error

This is the first command in the tutorial https://docs.microsoft.com/en-us/azure/hdinsight/spark/apache-spark-jupyter-spark-sql

dmueller1607 commented 4 years ago

Same issue for me, I'm using SparkMagic 0.14.0, Livy 0.5.0 and Spark 2.3.2. Is there a compatibility list for SparkMagic?

achidlow commented 4 years ago

This happened to me after seeing a timeout when creating the Livy SparkContext. It looks like other commands might still succeed if livy_session_startup_timeout_seconds is exceeded, but internally in sparkmagic things will not be in a good state, like for example fetching the sql_context_variable_name might have failed. Not sure if that is the issue anyone above has run into or if this is separate, but for anyone else landing here, either restarting the session after getting something like: LivyClientTimeoutException: Session 701 did not start up in 60 seconds. and/or increasing the timeout seconds if it happens regularly might help.

ETA: this was with sparkmagic 0.16.0, Livy 0.5.0 and Spark 2.3.2