fastplotlib demos for caiman notebooks

[kushalkolar]

A list of things that we should finish on the fastplotlib side, and ideas for how to use it in caiman demos without directly exposing all of the fastplotlib API considering its fast evolution. And the specific places it can be used.

Things to finish on fastplotlib side:

• [x] ~https://github.com/kushalkolar/fastplotlib/issues/255~ We have patched this using a DummyArray in mesviz, you can just use that if you want to display xy and txy data together in a ImageWidget
• [x] https://github.com/fastplotlib/fastplotlib/pull/371
• [x] https://github.com/kushalkolar/fastplotlib/issues/166

Places to use in caiman demos:

• playing movie
• viewing RCM, RCB, residuals. Should we also consider incorporating the LazyArrays for CNMF within caiman itself, the visualization of these is not viable without LazyArrays.
• interactive component viewing with movies, contours, single components, and heatmap.

For all of these, don't directly expose the entire fastplotlib API in the notebooks. Use ipywidgets. For example, a Int entry box for window functions with ImageWidget.

[EricThomson]

This is an important issue, and one probably easiest addressed via discussions on zoom/in-person. I think less important than the details about demos right now are big-picture questions, like at what point to make fastplotlib (fpl) a dependency? Should caiman have it as a dependency, or should we wait until we split caiman into a caiman-core/caiman-noncore (whatever that is called)?

[EricThomson]

Based on discussions with @pgunn it seems having this all be conda installable would go a long way.

I'm at the point where there are things that I really want to do in the demos, and there is no other tool that can do it. At this point I envision it like what we were doing with OpenCV with plotting toggles that can be easily set (e.g., in demo notebooks we currently have display_movie toggle because Qt).

Once the current workshop season is over, it would be nice to move this stuff up the queue and really start to integrate it (e.g., with some kind of judicious use of toggle that can easily be set). Then, next year we can unleash it on the world, this stuff is too good to not put in user's hands. It's like being in a clinical trial where one drug is clearly better than placebo and the trial needs to be stopped because it is unethical. :smile:

[kushalkolar]

ImageWidget tests are done :partying_face: :rocket: https://github.com/fastplotlib/fastplotlib/pull/371 , so you're good to go to start developing stuff. We have 29 test images that are cross-checked for ImageWidget functionality alone. You can even use the amazing test framework that Caitlin built :tada: to always validate that your notebook visualizations actually run properly! https://github.com/fastplotlib/fastplotlib/blob/main/examples/notebooks/image_widget_test.ipynb