Closed cometta closed 1 year ago
Hello,
try %%cleanup -f
. We do not use this kind of functionality so it's hard to say for sure.
cleanup -f
will delete all sessions including other people session. reference: https://github.com/jupyter-incubator/sparkmagic/blob/master/examples/Pyspark%20Kernel.ipynb . I look for way to delete my current session
Not the prettiest solution, but you could simply try this:
spark.stop()
exit()
This will shutdown the current application.
it works but the k8s session pod in kubectl describe
State: Terminated
│ Reason: Error
│ Exit Code: 1
│ Started: Fri, 14 Jul 2023 17:24:19 +0800
│ Finished: Fri, 14 Jul 2023 17:27:40 +0800
│ Ready: False
I don't think there is a way to exit gracefully. Also this feature should probably be developed on Sparkmagic side.
any spark magic command that i can run on last notebook cell to call lighter to shut down current running spark session and shut down current session k8s pods?