BUG: Bugs with VTK 9

[larsoner]

@GuillaumeFavelier you can now pretty easily build and install VTK 9 (which is good because they are on RC3, and it should fix some bugs and bring 3.8 support):

git clone --recursive https://gitlab.kitware.com/vtk/vtk.git
mkdir vtk-build
cd vtk-build
cmake -GNinja -DVTK_BUILD_TESTING=OFF -DVTK_WHEEL_BUILD=ON -DVTK_PYTHON_VERSION=3 -DVTK_WRAP_PYTHON=ON ../vtk
ninja -j 4
python setup.py bdist_wheel
pip install dist/vtk-*.whl

However, there is at least one rendering bug (depth peeling again maybe?) where the surface overlay gets cut off at some distance based on rotation (note the rightward "v" cut of the overlay):

We might need to figure out how to properly resolve the coincident topology here -- relying on VTK to do it automagically might not work anymore. Or maybe it's something else...?

[larsoner]

FWIW, PyVista worked "out of the box" this way, while Mayavi fails to compile (which I'll look into) :)

[larsoner]

Can someone else on macOS see if you can replicate https://gitlab.kitware.com/vtk/vtk/-/issues/17846 ?

[GuillaumeFavelier]

Thanks for starting this @larsoner. I'll follow the procedure you explained to be better prepared for VTK 9.

Also relevant for https://github.com/pyvista/pyvista/issues/562

[banesullivan]

Can you share that data or how to launch that scene? I'll try to look into this and see what's happening to the overlain mesh.

[larsoner]

WARNING: Will download 1.5 GB of data to ~/mne_data (unless you give it a different path to use)

import faulthandler
import os.path as op
import mne

faulthandler.enable()
data_path = mne.datasets.sample.data_path()
sample_dir = op.join(data_path, 'MEG', 'sample')
subjects_dir = op.join(data_path, 'subjects')
inv = mne.minimum_norm.read_inverse_operator(op.join(
    sample_dir, 'sample_audvis-meg-eeg-oct-6-meg-eeg-inv.fif'))
evoked = mne.read_evokeds(op.join(sample_dir, 'sample_audvis-ave.fif'))[0]
evoked.apply_baseline((None, 0))
stc = mne.minimum_norm.apply_inverse(
    evoked, inv, method='dSPM', verbose='debug', pick_ori=None)  # 'vector')
initial_time = 0.1

with mne.viz.use_3d_backend('pyvista'):
    brain = stc.plot(subjects_dir=subjects_dir, initial_time=initial_time,
                     clim='auto', views='lat', hemi='lh',
                     smoothing_steps='nearest', verbose=True)

You will probably need to pip install nibabel https://github.com/mne-tools/mne-python/zipball/master first.

This is the code I was using (last I recall!), but for some reason now it hangs when I try it.

[larsoner]

(FYI I forgot a ninja -j 4 in the command list above, I've updated it now)

[GuillaumeFavelier]

Sorry I am late, I had to build everything from scratch and follow your instructions.

This is the result of: python -c "import pyvista; print(pyvista.Report())"

--------------------------------------------------------------------------------
  Date: Wed Apr 15 15:02:56 2020 CEST

                OS : Linux
            CPU(s) : 4
           Machine : x86_64
      Architecture : 64bit
       Environment : Python
        GPU Vendor : NVIDIA Corporation
      GPU Renderer : GeForce GTX 960M/PCIe/SSE2
       GPU Version : 4.5.0 NVIDIA 430.64

  Python 3.6.10 |Anaconda, Inc.| (default, Mar 25 2020, 23:51:54)  [GCC 7.3.0]

           pyvista : 0.24.1
               vtk : 9.0.0
             numpy : 1.18.2
           imageio : 2.8.0
           appdirs : 1.4.3
            scooby : 0.5.2
            meshio : 4.0.10
             PyQt5 : 5.14.1
           IPython : 7.13.0
--------------------------------------------------------------------------------

EDIT I created vtk-build into vtk source folder. My mistake.

[GuillaumeFavelier]

rendering bug (depth peeling again maybe?) where the surface overlay gets cut off at some distance based on rotation

I can also reproduce the bug with the camera angle :+1:

I also notice an issue with point picking but maybe this one is related to _TimeViewer

[larsoner]

git clone --recursive https://gitlab.kitware.com/vtk/vtk.git
mkdir vtk-build
cd vtk-build

after these three steps you should be in vtk-build, not vtk/vtk-build. Perhaps you had an extra cd vtk after the git clone line?

[GuillaumeFavelier]

~Yes but the sources are in vtk and not vtk-build right? It's just an empty folder.~

Perhaps you had an extra cd vtk after the git clone line?

Exactly.

[GuillaumeFavelier]

I disabled depth peeling explictly without success:

ren_win = self.plotter.ren_win
ren_win.SetAlphaBitPlanes(0)
renderer = self.plotter.renderer
print(renderer.GetLastRenderingUsedDepthPeeling())  # prints 0
renderer.SetUseDepthPeeling(0)

[GuillaumeFavelier]

Using offset like we did with Vispy does work for me, I'll open a PR:

mapper.SetResolveCoincidentTopologyToPolygonOffset()
mapper.SetRelativeCoincidentTopologyPolygonOffsetParameters(-4., -4.)

[larsoner]

vtk-build starts out empty, but the cmake line populates it. I wonder if your problems were due somehow to building in a subdirectory of the source directory, rather than a totally separate one

[larsoner]

What's interesting about the topology problem is that I don't see it on mayavi. But whatever makes it work I guess!

[GuillaumeFavelier]

I don't see it on mayavi.

Any idea how they deal with it?

[larsoner]

No idea

[larsoner]

VTK9 rc3 wheels are up so you can try @GuillaumeFavelier :

https://vtk.org/download/

For me the stc.py example we always runs just freezes, I think due in part to this:

https://github.com/pyvista/pyvista/pull/686

It occurred to me that we actually need to fix this soon because once VTK9 wheels land on PyPi and are incorporated by Anaconda, anyone who installs or upgrades MNE will be stuck with unusable plotting :(

Can you look into it?

[GuillaumeFavelier]

I'm on it

[GuillaumeFavelier]

@larsoner I installed it and I have an issue with picking:

Traceback (most recent call last):
  File "/home/guillaume/source/mne-python/mne/viz/_brain/_timeviewer.py", line 939, in on_pick
    cell = mesh.faces[cell_id][1:]
IndexError: invalid index to scalar variable.

But except from that, the app is completely interactive. All the sliders and tool bar buttons work and are linked correctly.

[larsoner]

Hmm, must be some issue with my system. What's your mne sys_info? I'll try your version of Qt.

[GuillaumeFavelier]

Platform:      Linux-4.19.114-1-MANJARO-x86_64-with-arch-Manjaro-Linux
Python:        3.6.10 |Anaconda, Inc.| (default, Mar 25 2020, 23:51:54)  [GCC 7.3.0]
Executable:    /home/guillaume/source/anaconda3/envs/pyvista-dev/bin/python
CPU:           : 4 cores
Memory:        Unavailable (requires "psutil" package)
mne:           0.21.dev0
numpy:         1.18.3 {blas=openblas, lapack=openblas}
scipy:         1.4.1
matplotlib:    3.2.1 {backend=Qt5Agg}

sklearn:       Not found
numba:         Not found
nibabel:       Not found
cupy:          Not found
pandas:        Not found
dipy:          Not found
mayavi:        Not found
pyvista:       0.24.1
vtk:           9.0.0

[GuillaumeFavelier]

And I use PyQt5 5.14.1

[larsoner]

Ahh right without mayavi it will not tell you, let me update sys_info...

[larsoner]

Hooray! With pip install --upgrade --user pyqt5 which gives 5.14.2 everything works fine

[GuillaumeFavelier]

Feel free to try https://github.com/mne-tools/mne-python/pull/7665 then. It worked for me.

[larsoner]

@GuillaumeFavelier looks like the topology problems are still there:

https://mne.tools/dev/auto_examples/inverse/plot_source_space_snr.html#sphx-glr-auto-examples-inverse-plot-source-space-snr-py

[larsoner]

@GuillaumeFavelier might be good to fix this before looking into coreg

[GuillaumeFavelier]

For future reference, the visual issues mentioned in https://github.com/mne-tools/mne-python/issues/7599#issuecomment-692696125: