PR: Add wrappers to register methods of spyder-kernels by external plugins

[impact27]

The idea is to let external plugins expand spyder-kernels by sending source code to a kernel.

• A setup source code could be used to modify the kernel as needed
• A list of comm handlers can be registered

Spyder extensions would call register_external_plugin in ShellConnectMainWidget.create_new_widget and unregister_external_plugin in ShellConnectMainWidget.close_widget

The imports would be placed in setup_code

get_ipython() is imported automatically as a convenience

you can use inspect.getsource to get the source of functions if you don’t want to have strings of code in the source code

[jitseniesen]

This may or may not be related to this PR, but the spyder-unittest plugin needs to know whether specific Python modules are installed in the target environment (the environment for running Python code). Any idea what the best way is to do this?

[impact27]

You could register a callback using this PR to try to import the module and report on the success. Maybe importlib is also an option

[impact27]

@jitseniesen In the plugin (but here in spyder internal console):

>>> code = """
... import importlib
... def check_module_installed(module):
...     return importlib.util.find_spec(module) is not None
... """
>>> sw = spy.window.ipyconsole.get_current_shellwidget()
>>> sw.call_kernel(blocking=True).register_external_plugin(code, ["check_module_installed"])
>>> sw.call_kernel(blocking=True).check_module_installed("spyder")
True
>>> sw.call_kernel(blocking=True).unregister_external_plugin("", ["check_module_installed"])

[rear1019]

Using call_kernel(blocking=True) from within ShellConnectMainWidget.create_new_widget() fails for me (see backtrace below). Using interrupt=True instead works. (I haven’t investigated any further other than trying interrupt instead of blocking.)

Traceback (most recent call last):
  File "/home/reit/dev/_upstream/spyder/spyder/api/shellconnect/mixins.py", line 74, in add_shellwidget
    self.get_widget().add_shellwidget(shellwidget)
  File "/home/reit/dev/_upstream/spyder/spyder/api/shellconnect/main_widget.py", line 88, in add_shellwidget
    widget = self.create_new_widget(shellwidget)
  File "/home/reit/dev/_upstream/spyder-watchlist/spyder_watchlist/widgets/main_widget.py", line 69, in create_new_widget
    shellwidget.call_kernel(blocking=True).register_external_plugin(code, ["set_watchlist_expressions", "eval_watchlist_expressions"])
  File "/home/reit/dev/_upstream/spyder-kernels/spyder_kernels/comms/commbase.py", line 546, in __call__
    return self._comms_wrapper._get_call_return_value(
  File "/home/reit/dev/_upstream/spyder/spyder/plugins/ipythonconsole/comms/kernelcomm.py", line 221, in _get_call_return_value
    raise CommError("Cannot block on a disconnected comm")
spyder_kernels.comms.commbase.CommError: Cannot block on a disconnected comm

[impact27]

Maybe this is too early for blocking=True (the comm didn't have time to connect). You can simply use sw.call_kernel().register_external_plugin. The only problem is that in case of a problem no exception will be printed.

This error will become irrelevant anyway when https://github.com/spyder-ide/spyder/pull/16890 is merged (But this is for spyder 6)

[rear1019]

You can simply use sw.call_kernel().register_external_plugin. The only problem is that in case of a problem no exception will be printed.

Using call_kernel() with default arguments works.

[impact27]

Using call_kernel() with default arguments works.

Note that if you read a return value from the call, you need blocking=True or a callback

[impact27]

In general there are no ways of knowing what an external plugin might do. I simplified the PR further but now it is almost irrelevant. @ccordoba12 I will be closing it unless you think these changes are worth it. For example in @jitseniesen case, without this PR you need:

import importlib
kernel = get_ipython().kernel
def check_module_installed(module):
    return importlib.util.find_spec(module) is not None

kernel.frontend_comm.register_call_handler(
                "check_module_installed", check_module_installed)

VS this PR:

import importlib
kernel = get_ipython().kernel
@kernel.register_frontend_fun
def check_module_installed(module):
    return importlib.util.find_spec(module) is not None

Then in both cases you call the code with:

source = """
def register_plugin():
    // HERE THE CODE ABOVE
register_plugin()
del register_plugin
"""
shellwidget.execute(source, hidden=True)