Question: with and Output widget

[williamstein]

(Context: I'm writing a widget manager for CoCalc.)

Exactly why does this work fine in JupyterLab (and classic):

but this does not?

In particular, why does the backend kernel behave differently (and completely ignore the with block and not put output in the out widget's state) when used with one client (the console), but not with another client (jupyterlab or jupyter classic). I would have expected with to behave at the console more like the append_stdout method:

[jasongrout]

This difference in behavior is due to a design decision that output redirection occurs in the client, not the kernel. Basically, the output widget context manager sends a message to the frontend updating its msg_id attribute. The frontend output widget listens for changes on this attribute, and when it is set, it starts redirecting output at the frontend using whatever mechanism the frontend provides for redirecting output. The frontend output widget then gets the output messages that are redirected and sends them back to the kernel to update the outputs attribute. Since the console does not have a frontend output widget, no one sends the output messages back to the kernel.

This approach (of doing the redirection of output at the frontend, and reflecting that work back to the kernel for the output widget) shifts the burden of providing output redirection from the kernel author to the frontend author. The thinking here is that there will be far fewer frontends than kernels, and that redirecting output at the frontend is conceptually simpler (though it is also less capable when you start dealing with threaded execution). It does have the disadvantages of doubling the bandwidth needed for output widget outputs, and there are some subtle race conditions you have to be careful about (where the kernel output widget may not be updated with outputs until after the current frame of execution, after it processes the widget sync messages from the frontend). Also note that the other output redirection mechanism in Jupyter (display update messages) also does the routing of messages to where they are displayed in the frontend.

This is why when there are threads in your execution, we encourage people to directly use the append_* methods to directly manipulate the outputs attribute: https://ipywidgets.readthedocs.io/en/stable/examples/Output%20Widget.html#Interacting-with-output-widgets-from-background-threads

I imagine we will reconsider this when moving to a more centralized data model, where widget state is synced in a single store in the server, like what you are exploring.

[jasongrout]

For reference, in the classic notebook, we set this outputs attribute here: https://github.com/jupyter-widgets/ipywidgets/blob/367d3e193bd38593e3316b9420ec729bdc56f221/widgetsnbextension/src/widget_output.js#L40-L49 and in the jupyterlab manager, here: https://github.com/jupyter-widgets/ipywidgets/blob/367d3e193bd38593e3316b9420ec729bdc56f221/packages/jupyterlab-manager/src/output.ts#L102-L112

[williamstein]

Thanks! I would have never guessed that this is how things work...

In any case, that's enough of an explanation, and I've implemented this in CoCalc now. I did it so that realtime sync works, and everything happens on the backend with no messages going back and forth.

I think the right place for this processing is in the "kernel server", which for me is a node.js process, and for you is the Jupyter server. That avoids having to send the output back and forth (which feels like a hack), and also avoids the kernel itself having to deal with this.

[jasongrout]

Great! If we had had document and widget state on the server, that's where I would have implemented this. You get the best of both worlds, keeping the complications out of the kernel and the frontend. We'll be able to do this when we move state server-side.

[jasongrout]

Just curious: can you handle nested output widget context managers?

[williamstein]

Just curious: can you handle nested output widget context managers?

"In theory", yes. In practice the code I've wrote already fails to. I didn't think about that case and took a shortcut in the implementation to make things easier which makes it work incorrectly.

[williamstein]

Basically, the output widget context manager sends a message to the frontend updating its msg_id attribute.

Warning to anybody else who reads this -- for the record, sometimes this msg_id starts with execute_ and sometimes it doesn't. This caused me a lot of confusion. For example, if you use with explicitly in a cell, the msg_id has an execute_ prefix, but when with is used more implicitly (e.g., in interact), then it doesn't. I'm sure there is a good reason for this...

[williamstein]

@jasongrout I liked the challenge, so I implemented nested output. My test case was:

from ipywidgets import *
out1 = Output()
out2 = Output()
print('OUT1:')
display(out1)
print('OUT2:')
display(out2)

and then

with out1:
    print('xxx (under out1)')
    with out2:
        print("yyy (under out2)")
    print("zzz (under out1)")

This works fine now in cocalc-jupyter and also in classical jupyter.

The next obvious fun thing to try is:

with out1:
    print('xxx (under out1)')
    with out1:
        print("double nested")
    print("still here?")

and this FAILS. It fails in both my cocalc-jupyter implementation and in classical Jupyter:

It seems like the reason is because when the out1 backend widget is already capturing, it doesn't enable capturing again. Maybe capturing or not needs to be representing using a counter as well, not just a "yes/no" state. This seems like a bug in the implementation of widgets itself (the part that runs in the kernel) and/or a shortcoming in the protocol.

[jasongrout]

Thanks for finding this. I'll take a look.

Just curious, does widget rendering work in your capturing? How about multiple views of the output widget?

from ipywidgets import *
out1 = Output(layout={'border': '1px solid red'})
out2 = Output(layout={'border': '1px solid blue'})
display(out1)
display(out2)
with out1:
    display(IntSlider(description="In out1"))
    display(out2)
    with out2:
        display(IntSlider(description="In out2"))
    print('back in out1')

[williamstein]

Just curious, does widget rendering work in your capturing?

No, and I don't know why it doesn't (I'm surprised). I've added your example to my todo list.

How about multiple views of the output widget?

Yes, that works very well, even with multiple users at once.

[jasongrout]

even with multiple users at once.

Nice!

[jasongrout]

This seems like a bug in the implementation of widgets itself

Indeed! Fixed in #2384. Thanks for finding this!

[jasongrout]

Warning to anybody else who reads this -- for the record, sometimes this msg_id starts with execute_ and sometimes it doesn't. This caused me a lot of confusion. For example, if you use with explicitly in a cell, the msg_id has an execute_ prefix, but when with is used more implicitly (e.g., in interact), then it doesn't. I'm sure there is a good reason for this...

That's surprising to me. It should be using whatever the message id of the request_execute message was. I'm pretty sure we don't prepend execute_ in the classic notebook or jlab - is that where you were seeing it? Do you prepend execute_ sometimes in cocalc to your request_execute messages?

[jasongrout]

Here is where the msg_id comes from:

https://github.com/jupyter-widgets/ipywidgets/blob/f78dcb6474e68d7fed6202dd2d1ddd3fdecc835e/ipywidgets/widgets/widget_output.py#L110

It just uses whatever ipython thinks is the current parent header msg id for the request.

[williamstein]

Do you prepend execute_ sometimes in cocalc to your request_execute messages?

Woah -- you're right -- I do! So that mystery is solved.

[williamstein]

And now widgets in output widgets work:

I just had to pass down some props.

[jasongrout]

That was fast!

I guess we have some different default margins or padding compared to you, looking at the differences in the screenshots (my blue box is indented slightly more than yours, I think).

[williamstein]

I guess we have some different default margins or padding compared to you, looking at the differences in the screenshots (my blue box is indented slightly more than yours, I think).

I implemented my Output widget from scratch, and didn't even look at what the Phosphor implementation is like yet or worry about style. I do at least support passing in styles, so this example works and shows a border:

Why doesn't it work (the border) in the official docs?

[jasongrout]

Closing as answered, but I did open #2389 to solve that output border issue you noticed in the rendered docs.

[jasongrout]

CC also another output widget discussion with William, which contains a few more insights into the output widget design decisions: #2385