RFC scope / vision / notebook and more

[agramfort]

dear pyvista folks,

first I would like to tell you how pleased I am to see this package taking shape and attention from the community. We are with @GuillaumeFavelier now supporting it in MNE-Python. Our current dev version is now using it to render all our 3D plots https://mne.tools/dev/auto_examples/index.html For us it's a big step forward after trying many alternatives that turned out to be too fragile or too complex to maintain.

Now i would like to raise some concerns about the future and ask a few questions regarding notebook integration.

What makes pyvista appealing is what I would call "vtk for humans" or "vtk without learning the machinery". This comes with a simple API, a narrow and clear scope and a limited set of dependency (basically just vtk). This is by the way how we (re)built scikit-learn now about 10 years ago.

Questions:

• what is the scope of pyvista? basically where do you guys stop?

• how do we make it so that we have (good enough) notebook integration without exploding the list of dependencies. For example panel would pull nodejs ! and itkwidgets pulls itk while I would imagine that it is possible to have a "simple" vtkjs integration that would "just" serialize the vtk objects as compressed blobs, deserialize in js and then rely on vtkjs. If the scope of pyvista is rendering without a posteriori update of the properties of geometric objects (think traits) I have the impression that it should be doable with minimal dependencies.

Future and community effort: The notebook integration seems to be done in parallel by different teams without clear community vision. Mayavi can do this (I don't know how), panel offers this, itkwidgets etc. Are they all rewriting their serialization etc... code + some glue code in JS with widgets? How can we make it happen that more code is shared?

here are just some public thoughts that I hope will help us consolidate an open source ecosystem that appears to me still too fragmented.

my 2c Alex

[banesullivan]

First, it is so exciting that you all are using PyVista as a core piece of the MNE project for 3D rendering! It is definitely our biggest downstream dependency, and we've been happy to have @GuillaumeFavelier on board as a regular PyVista contributor!

What makes pyvista appealing is what I would call "vtk for humans" or "vtk without learning the machinery." This comes with a simple API, a narrow and clear scope, and a limited set of dependency (basically just vtk).

This perfectly captures our goal for the project - to be an intuitive and lightweight tool to help domain scientists get up and running with 3D visualization. I may borrow your "VTK for humans" quote 😄

what is the scope of pyvista? basically where do you guys stop?

This is an excellent question, and something that I don't think @akaszynski and I have discussed in detail. Moving forward in this post, I'll speak on my goals/ambitions for the project, but do note that I am not the only driving force here

I envision PyVista as one day being the go-to tool for 3D visualization and mesh analysis in Python (an ambitious goal). And as the "go-to" tool, it must have a ridiculously simple installation process (lightweight/minimal dependencies that won't conflict with other packages), and it must be intuitive to use for new users/novice programmers (no complicated routines). With those two goals, we will definitely have to start saying no to new feature requests that could overly complicate the API such as time-varying meshes like #412 (not that I am saying no to that, but we may need to evaluate how complicated that and other feature requests might make things and if those would diverge from our goals).

And so as we start to realize that we can't/shouldn't support everything around 3D visualization and mesh processing, we may need to move more towards viewing PyVista as an entry-level toolset for 3D visualization and mesh analysis rather than as a full-fledged suite. I am beginning to see PyVista as more of a tool to help researchers get up and running with VTK rather than be their end-all solution for 3D viz. For example, users can start with PyVsita and learn the ins/outs of 3D visualization, filtering, and analysis and then make things more and more sophisticated using the pure VTK python bindings intermixed with PyVista - which is what I do fairly often these days. Also, PyVista is great way to rapidly explore a workflow with a small dataset or quickly get some insight about a spatial dataset, but as soon as you have really large data or complicated analysis to perform, it may make sense to use PyVista as a prototyping tool, then go to ParaView when it's time to run your full workflow.

As users need more and more complex features, it may make sense to leave PyVista in its current state and have downstream projects that alter the API/provide wrappers/tools to do those more complex things like time-based mesh analysis. just a thought

how do we make it so that we have (good enough) notebook integration without exploding the list of dependencies.... it should be doable with minimal dependencies.

You're absolutely right here - we need to make sure that notebook rendering (inevitably in PyVista's future) adhere's to our goals of being easy to set up and use. I think itkwidgets is the clear choice as far as what tools to use moving forward, but agree that it needs work to become separated from the ITK project - I don't actually think ITK is needed as a dependency, it is just there because it is an ITK subproject.

the notebook integration seems to be done in parallel by different teams without clear community vision. Mayavi can do this (I don't know how), panel offers this, itkwidgets etc. Are they all rewriting their serialization etc... code + some glue code in JS with widgets? How can we make it happen that more code is shared?

All these efforts are separated at this time - panel is doing a great job of offering a serializer and making it easy to use a custom serializer, but I don't think this is a sustainable way forward. itkwidgets operates a bit differently and I think we should all focus around that framework as it is more easily extendable. One major advantage for panel though is its ability to save the whole embedded scene in the notebook so that you can host notebooks on GitHub like these:

• https://nbviewer.jupyter.org/github/OpenGeoVis/omfvista/blob/master/Example.ipynb
• https://nbviewer.jupyter.org/github/pyvista/show-room/blob/master/seg-eage-3d-salt-model.ipynb

[akaszynski]

@agramfort & @banesullivan

Thanks for your kind words. It's great to see other projects using pyvista!

Notebook Support

I'm envisioning that we create a separate module to encapsulate the notebook features as it is indeed a separate feature that's becoming more critical as universities and businesses have started to adopt Jupyter notebooks. Case in point, I'm using a branch of pyvista for notebook plotting using itkwidgets and panel. It's super helpful to have it right there with pyvista, but I think including it in the core module will lead to feature creep that will bloat pyvista.

@banesullivan: I propose we create pyvista_notebook (or pick a better name) and start adding features there to include several of the plotting backends. I've noticed that itkwidgets does not render on Apple mobile products (i.e. iPhone, iPad, etc), while it's fully supported by Samsung. Conversely, I've had problems with panel rendering on Windows. Since these are relatively new modules, we should support a few of them and as they develop, we can start to focus on the leading module.

By having a separate notebook module, we can ensure that pyvista is still easy to install and still has basic support for notebook plotting without bloating the core module. We can then direct users to install the notebook module should they wish for interactive notebook plotting.

Scope

When this project started forever ago as vtki, I never envisioned that the open source community would adopt pyvista at such a large scale. Now that the community is using the software, we should continue to adapt to their needs. I think VTK overstepped itself when it grew beyond just plotting to include boolean operations, math, ray tracing, mesh operations, and other advanced features that seem outside the scope of just being a "Visualization toolkit". We should support some of these features, but not strive to become a vtk scipy or other such application. pyvista should focus on 3D visualization, just as matplotlib is so great at 2D.

Should we wish to extend beyond visualization, we should develop separate modules that include those features rather than developing those features. In my opinion, this is limitation of C++ and one of the biggest features of Python: modularization. I'm in total agreement with @banesullivan; focus on what pyvista and vtk are good at, visualization.

[agramfort]

+100 for modularization and keeping limited and clear scopes for the different components. Keep dependencies limited is key to the large impact of pyvista

[banesullivan]

Closing due to staleness