3dplotting doesn't work

[bloyl]

I'm not sure how best to get 3d plotting to work.

currently mne sys_info return

$docker run -it mne_docker mne sys_info

Platform:      Linux-5.8.0-45-generic-x86_64-with-glibc2.10
Python:        3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 16:22:27)  [GCC 9.3.0]
Executable:    /opt/conda/bin/python
CPU:           x86_64: 4 cores
Memory:        Unavailable (requires "psutil" package)
mne:           0.22.0
numpy:         1.20.1 {blas=NO_ATLAS_INFO, lapack=lapack}
scipy:         1.6.1
matplotlib:    3.3.4 {backend=Qt5Agg}

sklearn:       0.24.1
numba:         Not found
nibabel:       3.2.1
nilearn:       0.7.1
dipy:          Not found
cupy:          Not found
pandas:        1.2.3
mayavi:        Not found
pyvista:       0.29.0 {OpenGL 3.3 (Core Profile) Mesa 18.3.1 via llvmpipe (LLVM 7.0, 256 bits)}
vtk:           9.0.0
PyQt5:         5.15.4

and

$ docker run mne_docker python -c "import pyvista; pl=pyvista.Plotter(); print(pl.ren_win.ReportCapabilities())"

OpenGL vendor string:

and finally,

$ docker run -it mne_docker python work/plot_eeg_on_scalp.py
Opening raw data file /home/jovyan/mne_data/MNE-sample-data/MEG/sample/sample_audvis_raw.fif...
    Read a total of 3 projection items:
        PCA-v1 (1 x 102)  idle
        PCA-v2 (1 x 102)  idle
        PCA-v3 (1 x 102)  idle
    Range : 25800 ... 192599 =     42.956 ...   320.670 secs
Ready.
Using outer_skin.surf for head surface.
Using pyvista 3d backend.

qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, wayland-egl, wayland, wayland-xcomposite-egl, wayland-xcomposite-glx, webgl, xcb.

similar code run in juypter crashes the kernel.

[bloyl]

I'm thinking about trying to rebuild mesa with opengl drivers but I'm not sure that will help.

[GuillaumeFavelier]

What happens if you change the 3d backend to notebook with MNE_3D_BACKEND=notebook and set pyvista offscreen with PYVISTA_OFF_SCREEN=true? The goal would be to make it avoid using PyVistaQt but just PyVista.

[GuillaumeFavelier]

Argh, it won't work either because it's not an ipython environment:

(base) jovyan@820c740390a6:~$ MNE_3D_BACKEND=notebook python work/plot_eeg_on_scalp.pyOpening raw data file /home/jovyan/mne_data/MNE-sample-data/MEG/sample/sample_audvis_raw.fif...
    Read a total of 3 projection items:
        PCA-v1 (1 x 102)  idle
        PCA-v2 (1 x 102)  idle
        PCA-v3 (1 x 102)  idle
    Range : 25800 ... 192599 =     42.956 ...   320.670 secs
Ready.
Using outer_skin.surf for head surface.
Using notebook 3d backend.

Traceback (most recent call last):
  File "work/plot_eeg_on_scalp.py", line 22, in <module>
    fig = plot_alignment(raw.info, trans, subject='sample', dig=False,
  File "<decorator-gen-141>", line 24, in plot_alignment
  File "/opt/conda/lib/python3.8/site-packages/mne/viz/_3d.py", line 1014, in plot_alignment
    renderer = _get_renderer(fig, bgcolor=(0.5, 0.5, 0.5), size=(800, 800))
  File "/opt/conda/lib/python3.8/site-packages/mne/viz/backends/renderer.py", line 38, in _get_renderer
    return backend._Renderer(*args, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/mne/viz/backends/_notebook.py", line 17, in __init__
    ipython.magic('matplotlib widget')
AttributeError: 'NoneType' object has no attribute 'magic'

[GuillaumeFavelier]

From discourse:

To be able to use non interactive scripts to do analysis, which includes off screen 2d/3d plotting to files and to mne.Report objects. I think this is identical to a true headless HPC type of install.

I don't know what you expect from this but I played a bit with docker and this is what I found.

First, I created a prototype branch of mne-python with a "toy" offscreen 3d backend. For simple scripts (non-Qt scripts to be exact), it just creates a screenshot when the show() function is called using _get_screenshot_filename():

https://github.com/GuillaumeFavelier/mne-python/tree/proto/off_screen

Example:

MNE_3D_BACKEND=offscreen PYVISTA_OFF_SCREEN=true python work/plot_eeg_on_scalp.py

Then, after ls:

environment.yml    MNE_2021-03-23_14-39-13.png  server_environment.yml  work
labextensions.txt  mne_data                     this_env.yml

MNE_3D_BACKEND to select the backend and PYVISTA_OFF_SCREEN to force pyvista to close when show instead of creating an interactive window. But using the default env created by the Dockerfile, the screenshot was not generated though so there is something wrong somewhere. I managed to get something only when I used a fresh new conda env with pip-installed mne (my branch), pyvista, matplotlib and vtk (I wanted to fetch 9.0.1).

Platform:      Linux-4.19.180-1-MANJARO-x86_64-with-glibc2.10
Python:        3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 16:22:27)  [GCC 9.3.0]
Executable:    /opt/conda/envs/mne/bin/python
CPU:           x86_64: 4 cores
Memory:        Unavailable (requires "psutil" package)
mne:           0.23.dev0
numpy:         1.20.1 {blas=openblas, lapack=openblas}
scipy:         1.6.1
matplotlib:    3.3.4 {backend=TkAgg}

sklearn:       Not found
numba:         Not found
nibabel:       Not found
nilearn:       Not found
dipy:          Not found
cupy:          Not found
pandas:        Not found
mayavi:        Not found
pyvista:       0.29.0 {OpenGL 4.5 (Core Profile) Mesa 20.2.6 via llvmpipe (LLVM 11.0.0, 256 bits)}
vtk:           9.0.1
PyQt5:         Not found

Maybe it would work with just environment.yml but I did not check.

My goal was to prototype a typical offscreen workflow with mne. I could work more on this on mne-python and think about a better integration if we discuss it there directly. I could link my branch to your issue in a PR :+1:

[bloyl]

@GuillaumeFavelier Thank you for looking at this. I will hopefully play with this some tonight and see how far I get. Its great to see that the notebook backend already takes Qt out of the loop.

The idea of an offscreen mne_3d_backend, that automatically renders to disk is interesting but wasn't quite what I was thinking. In my head, I was thinking more along the lines of adding figures to an mne.report or calling the screen_shot methods from top level scripts. That way the end user would have full control over output file names as well as the ability to tweak the plots post the standard MNE plotting.

[GuillaumeFavelier]

For Report, it is possible to use add_image or add_slider. There is an example for Brain plots:

https://github.com/mne-tools/mne-python/pull/8730

Other plots should return a fig. It's possible to get a screenshot with fig.plotter.screenshot() for example but this is not officially supported (API still under development).

This is still too painful for users IMO so I see two main paths from here:

1) Make it work with PyVistaQt in offscreen mode (I need to investigate seriously) 2) Decouple PyVista from Qt and use the basic screenshot features (basically what I tried in https://github.com/GuillaumeFavelier/mne-python/tree/proto/off_screen)

[bloyl]

@GuillaumeFavelier : Take a look at the https://github.com/bloyl/mne_docker/tree/test_3d branch.

After a number of different iterations, i think I have something that is close to working for using the full pyvistaqt backend

if you do the following.

mkdir outputs
chmod 777 -R outputs/
docker build -t mne_alpine -f Dockerfile .

docker run -it -vpwd/outputs/:/outputs mne_alpine python mne_test/test_pyvista.py runs and saves an image of a sphere in the outputs directory.

However docker run -it -vpwd/outputs/:/outputs mne_alpine python mne_test/plot_eeg_on_scalp.py runs but creates a blank image in the outputs directory.

So It seems like the setup is correct and working since the pyvista plotting is good, but something is off on the mne side of things or i'm missing something in generating a figure.

Any ideas what to try next?

[GuillaumeFavelier]

I'm still building the docker image but here are my first thoughts:

I checked test_pyvista.py:

import pyvista as pv

sphere = pv.Sphere()

plotter = pv.Plotter()
plotter.add_mesh(sphere)
plotter.show(screenshot='/outputs/pyvista_out.png')

It uses pyvista.Plotter which is a good test for vtk and pyvista so it's definitely good news but it does not use pyvistaqt.

Here is what I would suggest as a test_pyvistaqt.py (to obtain a better representation of what is used in MNE):

import pyvista as pv
from pyvistaqt import BackgroundPlotter  # <--- this uses Qt here

sphere = pv.Sphere()

plotter = BackgroundPlotter()
plotter.add_mesh(sphere)
plotter.screenshot('/outputs/pyvistaqt_out.png')

[bloyl]

Funnily enough I was just playing with this. And did exactly what you suggest. let me push what I have.

Interestingly. The numpy array returned by .screenshot is non zero in the pyvistaqt plots but in the mne plots it's all zero.

On Fri, Apr 30, 2021, 10:11 AM Guillaume Favelier @.***> wrote:

I'm still building the container but here are my first thoughts:

I checked test_pyvista.py:

import pyvista as pv sphere = pv.Sphere() plotter = pv.Plotter()plotter.add_mesh(sphere)plotter.show(screenshot='/outputs/pyvista_out.png')

It uses pyvista.Plotter which is a good test for vtk and pyvista so it's definitely good news but it does not use pyvistaqt.

Here is what I would suggest as a test_pyvistaqt.py (to obtain a better representation of what is used in MNE):

import pyvista as pvfrom pyvistaqt import BackgroundPlotter # <--- this uses Qt here sphere = pv.Sphere() plotter = BackgroundPlotter()plotter.add_mesh(sphere)plotter.screenshot('/outputs/pyvista_out.png')

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/bloyl/mne_docker/issues/1#issuecomment-830122982, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABKTXHOJQAZOO5N56D6PU3TTLK3BLANCNFSM4ZPPEXYQ .

[GuillaumeFavelier]

I'll close #3 then :)

[bloyl]

ok so still on the test_3d branch

docker run -it -v `pwd`/outputs:/outputs mne_docker python mne_test/test_pyvistaqt.py
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-mne_user'
min:73 - max:237

and the mne plot doesn't work

docker run -it -v `pwd`/outputs:/outputs mne_docker python mne_test/plot_test.py
Using pyvista 3d backend.

QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-mne_user'
min:0 - max:0

Is your expectation that plot_test.py should work?

[GuillaumeFavelier]

Yes, basically if test_pyvistaqt.py works, I would expect plot_test to work as well :thinking:

[GuillaumeFavelier]

Once I have working container, I'll try to understand what is going on :+1:

(For now I'm stuck at the downloading data step)

[bloyl]

are you building the test_3d branch? that should be pretty fast to build.