Can ui_events() be called from outside the main jupyter thread?

[NicoKiaru]

Disclaimer: I'm not an expert in python, so I hope my question makes sense.

I am trying to 'emulate' Java modal dialogs by using ipywidgets, this repo, and JPype.

I tested the example notebooks and everything is working fine (which is great because that's the first easy option I've found so far!).

However, things are failing when I try to use it in combination with JPype.

I believe, but I'm not sure, that the problem is linked to multithreading.

The Java code I use is creating new threads for the UI, and each java thread is linked a new python thread. Then, this new python thread is displaying a widget with an ok button. This works fine. I now want to wait for the user to click on the widget. I thus put this sort of code:

       self.user_has_clicked = False
       ok_button = widgets.Button(description='OK')
       ok_button.on_click(self.process_click)
       list_of_widgets.append(ok_button)
       logger.debug('Display widgets module '+str(module))
       display(widgets.VBox(list_of_widgets))

       with ui_events() as ui_poll:
            while not self.user_has_clicked:
                print(self.user_has_clicked, end="")
                ui_poll(10)  # Process a few events per iteration
                time.sleep(0.1)

And the code is failing at the line with ui_events() as ui_poll:. Worse: I can't collect any meaningful error message in jupyter. It thinks that java is failing.

In terms of thread, I think what's happening is this:

Thread 1: main python thread -> java main thread -> start java ui thread -> wait for end of java ui thread -> bug received -> return error to python
Thread 2:                                               java ui thread -> new python thread  -> show widget -> bug with ui_events()? 

I do not know how to debug this easily... I am using PyCharm for pure Python, which makes it easy to debug python, but I don't know how to do the same with jupyter.

• In short, is it safe to call ui_events() when started from another python thread ?
• Is there an easy scenario to test this ?

[NicoKiaru]

Ok, I give up for now.

It seems that asyncio.get_event_loop() crashes in

    def get() -> "KernelWrapper":
        if KernelWrapper._current is None:
            KernelWrapper._current = KernelWrapper(
                get_ipython(), asyncio.get_event_loop()
            )
        return KernelWrapper._current

[Kirill888]

@NicoKiaru this library is meant to be used inside the thread that is evaluating notebook cell, it forces ipykernel library to evaluate ui events (callbacks registered by ui libraries) while evaluating a notebook cell. It won't be safe to run that in a separate thread, nor is there a need if you just launch a thread and then finish execution of a cell, as ui events get processed as soon as they get generated when no cell is blocking execution.

I'm not familiar with JPype, does it integrate with ipython event loop at all?

In your case you should try this approach:

Main thread: launch java thread then enter ui_events loop in the main thread, when the result is populated in the java thread exit the main loop Java thread: do whatever ui setup/teardown you need, don't use this library at all.

[NicoKiaru]

Thanks for your answer!

I'm not familiar with JPype, does it integrate with ipython event loop at all?

There's a paragraph about threading, not specific to ipython here: https://jpype.readthedocs.io/en/latest/userguide.html#concurrent-processing

All of this looks a bit too involved for me. In the best case scenario I manage to write a small minimal example of what I want to achieve (in pure python) and post it here. But I don't know if and when I'll have time to do that. So feel free to close the issue.

[NicoKiaru]

Ah, I think I've found what would work for me:

modify the code in each jupyter cell in such a way that it's run in another thread, then make sure that the thread finishes with your code.

So typically, wrap each cell code like this:

this_cell_task = Thread(target=lambda _: inner_cell_code)
this_cell_task.start()
with ui_events() as poll:
        while this_cell_task.is_alive():
            poll(10)          # React to UI events (upto 10 at a time)
            time.sleep(0.1)

I've tested and it seems to work.

Do you think it's a reasonable idea ? Is there a possibility to 'magically' transform the code of each cell like that ?

[NicoKiaru]

This works as a magic command for instance:

from IPython.core.magic import register_cell_magic

@register_cell_magic('run_in_another_thread')
def run_in_another_thread(line, cell):
    try:
        from jupyter_ui_poll import ui_events
    except ImportError:
        return "'jupyter-ui-poll' not installed. Did you run 'pip install jupyter-ui-poll'?"
    from threading import Thread
    from IPython.core.interactiveshell import InteractiveShell
    shell = InteractiveShell.instance()
    def f():
        shell.run_cell(cell)
    cell_runner = Thread(target=f)
    cell_runner.start()
    with ui_events() as poll:
        while cell_runner.is_alive():
            poll(10)          # React to UI events (upto 10 at a time)
            time.sleep(0.1)

[Kirill888]

this looks fine as an approach. Whether to use magics vs code transform hooks vs library functions depends on your needs/preferences. I would probably go with the library approach, but magics are handy too.

[NicoKiaru]

I didn't know about hooks, I guess it can solve my problem at the notebook level instead fo solving it at the cell level.

I'll check this also.

Thanks a lot for this repo, I went from despair to hope ;-)