search

jupyter-incubator / sparkmagic

Jupyter magics and kernels for working with remote Spark clusters
Other
1.33k stars 447 forks source link

/reconnectsparkmagic doesn't work, doesn't seem to do anything, and is poorly documented #559

Open bschreck opened 5 years ago

bschreck commented 5 years ago

I was running into the same issues discussed in #286 , and thought that maybe the /reconnectsparkmagic API call is potentially a new feature that has since been implemented to handle this. However, the README is pretty sparse about what this does, and it didn't work off the bat. I was getting 404 errors, and I had to modify the source to make the endpoint '/api/reconnectsparkmagic' in serverextension/handlers.py:load_jupyter_server_extension.

Even with this modification, it doesn't seem like this extension does anything. And if it does, it's very unclear what it is. I managed to get the code to run, but I still can't connect to an existing spark session.

If it doesn't do anything, I would suggest removing it from the README. If it does, I would document better what that is.

itamarst commented 5 years ago

Thanks for the bug report!

yangyishan0901m commented 5 years ago

I observe similar issue on /reconnectsparkmagic, Jupyter 5.7.0 + sparkmagic 0.12.9 Assertion failed: rc == 0 (src/socket_poller.cpp:41) I am using jupyter docker stack, and /reconnectsparkmagic will break connection to jupyter container.