Prototype of ipyniivue using anywidget

[kolibril13]

Hi there! @jens-ox reached out to me on Twitter and asked for tips to connect nivvue to Jupyter.

I just had some time playing around with this library and came up with this prototype using anywidget and react: https://github.com/kolibril13/anywidget-ipyniivue (example notebook in repo) pip install anywidget_ipyniivue

https://github.com/niivue/ipyniivue/assets/44469195/08c3d466-1ca1-44cf-a57d-9e2203c3d335

Here's how the widget backend is defined: https://github.com/kolibril13/anywidget-ipyniivue/blob/0d39d180b1dcb1d9b139e11a914366174436e513/src/anywidget_ipyniivue/__init__.py#L13-L17 and here's how the frontend is defined: https://github.com/kolibril13/anywidget-ipyniivue/blob/main/js/widget.jsx

[hanayik]

@kolibril13 , this looks great! I just ran it in a notebook locally and it worked well :). I think this approach significantly simplifies the development of the jupyter notebook integration.

I think the tricky part will be waiting for asynchronous NiiVue functions (for example: loadVolumes).

Also, it would be useful to keep NiiVue setter/getter methods in sync with the python API. Maybe a useful approach would be to develop the jupyter notebook integration in the same repo as the niivue package. Then we could share config files etc that keep the python API and the Javascript API in sync as NiiVue development moves forward (exact implementation TBD). So instead of setting w.value = "https://niivue.github.io/niivue-demo-images/mni152.nii.gz" the Python API would match the NiiVue API and it could look like this: w.loadVolumes([{"url", "https://niivue.github.io/niivue-demo-images/mni152.nii.gz"}])

What do you think?

We have a NiiVue development group meeting on 10th Jan at 3pm (London TZ). You're welcome to join and we can add this to the discussion. Email me at taylor.hanayik@ndcn.ox.ac.uk for an invite.

[kolibril13]

I think this approach significantly simplifies the development of the jupyter notebook integration.

that's great to hear!

I think the tricky part will be waiting for asynchronous NiiVue functions (for example: loadVolumes).

Can you tell me more what you mean by waiting for asynchronous function would be the tricky part? This is probably something that can be investigated on the anywidget side by looking at a minimal example (e.g. async function with a simple timeout).

instead of setting w.value = "https://niivue.github.io/niivue-demo-images/mni152.nii.gz" the Python API would match the NiiVue API and it could look like this: w.loadVolumes([{"url", "https://niivue.github.io/niivue-demo-images/mni152.nii.gz"}])

sounds reasonable! Also, here it could help to make a minimal example with anywidget in order to figure out how to change a traitlet via function call.

NiiVue development group meeting on 10th Jan at 3pm (London TZ). You're welcome to join

Thanks for the invitation, I'm happy to join!

[christian-oreilly]

Some preliminary thoughts. I don't know this framework, so some of my assumptions may be wrong.

Looking at the code, it seems like it is a state-based system, where state variables are created and then synchronized automatically between the Python and the JS side. From Python's side, I assume the "on_change" callback from the traitlet variables get triggered when theit values are updated. So, it looks like, on the Python side, state variables are updated and the framework takes care of updating the JS side accordingly. It is not clear to me how this can be extended to the re-use of JS functions from the Python side. Wrapping JS-side functions in Python (rather than just mapping the base arguments) might be desirable to avoid having to recreate the logic of these functions on the Python side. I am not sure if doing so requires circumventing the mechanisms used by anywidget and will cause the sync issues to pop back.

@kolibril13 The issues with waiting for asynchronous function have not been discussed consistently in one single issue, but are are the roots of many of the issues currently open. Issue #38 is probably a good example.

[christian-oreilly]

@kolibril13 This post also captures well the issue: https://discourse.jupyter.org/t/synchronously-call-javascript-function-from-python/10059

[kolibril13]

it seems like it is a state-based system, where state variables are created and then synchronized automatically between the Python and the JS side.

yep, that's right! I'm also planning to make a tutorial on anywidget that gives some minimal examples how to use them. So far I've only made a tutorial for ipyreact (which is based on anywidget, so most things are similar: https://github.com/widgetti/ipyreact/blob/master/examples/full_tutorial.ipynb)

I assume the "on_change" callback from the traitlet variables get triggered when theit values are updated

exactly!

Wrapping JS-side functions in Python (rather than just mapping the base arguments) might be desirable to avoid having to recreate the logic of these functions on the Python side

Not sure I understand what you mean by this. Can you maybe give a minimal example?

[christian-oreilly]

Wrapping JS-side functions in Python (rather than just mapping the base arguments) might be desirable to avoid having to recreate the logic of these functions on the Python side

Not sure I understand what you mean by this. Can you maybe give a minimal example?

@kolibril13 I meant the same thing that @hanayik said, i.e., using a function call (w.loadVolumes([{"url", "https://niivue.github.io/niivue-demo-images/mni152.nii.gz"}])) instead of an attribute update (i.e., w.value = "https://niivue.github.io/niivue-demo-images/mni152.nii.gz").

[christian-oreilly]

Note for example that the current structure does not allow loading multiple volumes since the arguments of loadVolumes is built on JS side: https://github.com/kolibril13/anywidget-ipyniivue/blob/0d39d180b1dcb1d9b139e11a914366174436e513/js/widget.jsx#L11-L16