Merge @jupyterlab/debugger into JupyterLab core

[afshin]

We propose merging @jupyterlab/debugger into JupyterLab core as a 3.0 feature that only appears for users who have a kernel that supports debugging. Because the end-user experience is transparent, this feature should not cause disruption.

As of July 2020, the only kernel that supports the Debug Adapter Protocol is the xeus-python kernel, but we are eager for other kernels to support debugging. The front-end debugger is kernel-agnostic and will only show up for users who have installed at least one kernel that supports DAP.

cf. VSCode support for debugging Jupyter notebooks:

• June 2020 (version 1.47)
• debugging with xeus-python example

[afshin]

Because it's possible for the debugger to detect whether any kernels support debugging without any user action necessary, I would suggest we take advantage of that to provide the best automatic experience possible.

[isabela-pf]

From a UX perspective, it seems like a good idea to allow debugging for kernels that support debugging, so it's a yes vote from me.

I am also happy to help work out the experience and UI for this, particularly the experience for what the debugger does when no kernels that support debugging are installed since that seems to be one of the main concerns.

[lresende]

When I think as a JupyterLab Contributor having the debugger closest to core would provide the best visibility and facilitate the community to implement the necessary support into widely kernels.

When I put myself as a user, seeing a widely requested feature such as debugging capability available in JupyterLab, would tease me to use it, but I would quickly realize I can't debug my existing notebooks as is. IMHO, I still think that this should be enabled as part of the widely adopted/used kernels such as IPython kernel.

A compromise being discussed here involves remediation steps such as: clear documentation about how the debugger can be used, and the limitations around the kernel(s) that support the debugging protocol and also that, by default, this would be hidden from the user that does not have proper requirements to run the debugger.

Based on the compromises described above, I am +1 on merging the debugger into JupyterLab core.

[lresende]

cf. VSCode support for debugging Jupyter notebooks:

* [June 2020 (version 1.47)](https://code.visualstudio.com/updates/v1_47)

* [debugging with `xeus-python` example](https://github.com/microsoft/vscode-simple-jupyter-notebook)

Also, just a clarification on the topic of debugging notebooks with VSCode, my understanding is that it requires VSCode Insiders + the above github repository extension in order to enable the debugging UI with Xeus code, but when I followed the documented steps with the latest VSCode Insiders and Xeus release, the UX was ok, but actually debugging was causing multiple exceptions.

[jtpio]

Note that the latest release of @jupyterlab/debugger now hides the sidebar completely if no kernel supports debugging (which is the case with a stock install of JupyterLab at the moment).

It can be tested on Binder: https://mybinder.org/v2/gist/jtpio/27533014e25ca65c85d0bafff96f38b9/master?urlpath=%2Flab

(from this gist: https://gist.github.com/jtpio/27533014e25ca65c85d0bafff96f38b9)

Then install xeus-python and the sidebar will be displayed after the next page refresh:

[afshin]

Thanks for the screencast, Jeremy!

[jasongrout]

I support merging the debugger into core, especially with the debugger JEP basically being approved at this point, and the debugging protocol becoming a Jupyter standard.

[timkpaine]

Does xeus have this critical feature of ipykernel?

from ipykernel import IPKernelApp; IPKernelApp.launch_instance()

Without the ability to launch from an existing python process it doesn't have much utility for those of us with large, custom python deployments.

retracting this since xpython_extension seems to work fine in my environment! Just want to make sure we keep in mind the non-conda places, conda should never be a prerequisite to getting something working.

[SylvainCorlay]

retracting this since xpython_extension seems to work fine in my environment!

Yes, the xpython_extension makes xeus-python somewhat closer to ipykernel, but we aren't 100% sure yet if this should be the default way to launch it - but we have been iterating on streamlining that usecase, for the purpose of the PyPI wheel.

[SylvainCorlay]

Once enough polished, a second step would be to move that to a pure kernel repo. The first candidate I see would be to move it to jupyter-client, on a page like https://jupyter-client.readthedocs.io/en/stable/messaging.html.

The JEP has been merged, and the Jupyter Debugger Prototol is now described in https://jupyter-client.readthedocs.io/en/latest/messaging.html.

[afshin]

Closed in https://github.com/jupyterlab/jupyterlab/pull/8747