Include MNELAB

[hoechenberger]

Closes #82

Just want to see if it works

[hoechenberger]

The ARM build is expected to fail, as we're still waiting for the migration to complete on conda-forge

[larsoner]

All green except ARM as expected, great!

[hoechenberger]

If you have time, could you download and test the artifacts?

Also I think we need some sort of CI test for MNELAB

Mind you, I believe we now have Qt5 and Qt6 installed in parallel! Who knows what kind d of interactions might occur 😱

[larsoner]

We should just switch to Qt6, the rest of MNE should be compatible with it already

I'll test on my M1 once it's working there :)

[larsoner]

... actually I'm not sure if mayavi is compatible. We'd need to check for mne-kit-gui :(

[hoechenberger]

I don't think spyder supports Qt6 either

[hoechenberger]

I remember the conda-forge folks spending quite some time to ensure isolation between Qt5 and Qt6, so at least in theory things should work. But who knows how well downstream packages can handle the simultaneous presence of both

[hoechenberger]

The ARM build is now on conda-forge, restarting CI.

[hoechenberger]

Ouch, dependency resolution error 😳 this is going to be a pain to pin down 😑

[larsoner]

Locally on M1 when I try to make an env with our package list I get:

$ mamba env create -n test -f my-env.yml 
...
Encountered problems while solving:
  - package mnelab-0.8.5-py38h10201cd_0 requires python >=3.8,<3.9.0a0, but none of the providers can be installed

which is weird because the builds for other Python versions do show up in https://anaconda.org/conda-forge/mnelab/files . Maybe CDNs are just way behind or something?

[larsoner]

Looks like it's actually a conflict with vtk =9.2.2. With even just:

name: test
channels:
  - conda-forge
dependencies:
  - mnelab =0.8.5
  - python =3.10.8
  - vtk =9.2.2

I get:

...
UnsatisfiableError: The following specifications were found to be incompatible with each other:

Output in format: Requested package -> Available versions

Package python conflicts for:
vtk=9.2.2 -> loguru -> python[version='3.10.*|3.11.*|>=3.5|>=3.6|3.8.*|3.9.*']
mnelab=0.8.5 -> python[version='>=3.10,<3.11.0a0|>=3.10,<3.11.0a0|>=3.8,<3.9.0a0|>=3.8,<3.9.0a0|>=3.9,<3.10.0a0',build=*_cpython]
vtk=9.2.2 -> python[version='>=3.10,<3.11.0a0|>=3.10,<3.11.0a0|>=3.11,<3.12.0a0|>=3.8,<3.9.0a0|>=3.8,<3.9.0a0|>=3.9,<3.10.0a0|>=3.9,<3.10.0a0',build=*_cpython]
python=3.10.8
mnelab=0.8.5 -> matplotlib-base[version='>=3.6.2'] -> python[version='3.10.*|3.8.*|3.9.*|>=3.11,<3.12.0a0|>=3.7|>=3.6|>=3.5',build=*_cpython]

Package expat conflicts for:
vtk=9.2.2 -> expat[version='>=2.4.9,<3.0a0|>=2.5.0,<3.0a0']
vtk=9.2.2 -> python[version='>=3.9,<3.10.0a0'] -> expat[version='>=2.4.1,<3.0a0']

Package setuptools conflicts for:
python=3.10.8 -> pip -> setuptools
mnelab=0.8.5 -> mne-base[version='>=1.0.0'] -> setuptools

Package freetype conflicts for:
vtk=9.2.2 -> ffmpeg[version='>=4.4.2,<5.0a0'] -> freetype[version='>=2.10.4,<3.0a0']
vtk=9.2.2 -> freetype[version='>=2.12.1,<3.0a0']

Package libcblas conflicts for:
mnelab=0.8.5 -> numpy[version='>=1.20.0'] -> libcblas[version='>=3.9.0,<4.0a0']
vtk=9.2.2 -> numpy -> libcblas[version='>=3.9.0,<4.0a0']

Rolling back to vtk 9.1 doesn't help because then there are conflicts with openmeeg :(

[cbrnr]

Is there anything MNELAB can do to help solve this conflict? I'm not pinning any dependencies (only >=) because MNELAB should work always work with the latest versions. Python versions 3.8–3.11 should be supported (I'm aiming for the last three versions, right now these are even four).

[hoechenberger]

@larsoner I suspect it has to do with me pinning matplotlib to the latest version... But 3.6.2 is required to work around bugs in PySide 6.4 😩

https://github.com/cbrnr/mnelab/issues/362

[hoechenberger]

@larsoner I suspect it has to do with me pinning matplotlib to the latest version... But 3.6.2 is required to work around bugs in PySide 6.4 😩

cbrnr/mnelab#362

This assumption was incorrect. I can reproduce the issue by simply requesting vtk and pyside6.

❯ mamba create -n test pyside6=6.4.1 vtk=9.2                                                                     ─╯
[...]
Encountered problems while solving:
  - package pyside6-6.4.1-py310h6bbb1e2_0 requires libclang >=13.0.1,<14.0a0, but none of the providers can be installed

We should get in touch with the VTK feedstock maintainers.

[hoechenberger]

I reported it to the VTK and PySide6 feedstock maintainers:

https://github.com/conda-forge/vtk-feedstock/issues/269 https://github.com/conda-forge/pyside2-feedstock/issues/159

[larsoner]

Great @hoechenberger ! Now we wait ...

[hoechenberger]

@larsoner Could be a long time…

https://github.com/conda-forge/pyside2-feedstock/issues/159#issuecomment-1350932070

[hoechenberger]

I think we should simply build the M1 installers without MNELAB, as the ETA of the OpenSSL migration, which seems to be required to resolve the issue, is unclear (and probably a few months away)

[larsoner]

Works for me, feel free to take out of draft mode and mark for merge-when-green @hoechenberger

[hoechenberger]

@larsoner Are we sure the Windows installer problems @mscheltienne observed recently are gone now, too? I cannot test, unfortunately

[larsoner]

I tested locally on Windows last week and things were fine

[hoechenberger]

Great! Enabling automerge.

[hoechenberger]

I cannot run MNELAB on M1:

❯ mamba create -n test mnelab
❯ mamba activate test
❯ mnelab
2022-12-14 13:40:27.154 python[43193:310391] ApplePersistenceIgnoreState: Existing state will not be touched. New state will be written to /var/folders/2k/nkw3dkx97ssdv08xmdgfnjkc0000gn/T/com.continuum.python.savedState
Traceback (most recent call last):
  File "/Users/hoechenberger/mambaforge/envs/test/bin/mnelab", line 11, in <module>
    sys.exit(main())
  File "/Users/hoechenberger/mambaforge/envs/test/lib/python3.10/site-packages/mnelab/__init__.py", line 42, in main
    model.view = MainWindow(model)
  File "/Users/hoechenberger/mambaforge/envs/test/lib/python3.10/site-packages/mnelab/mainwindow.py", line 285, in __init__
    QKeySequence(Qt.CTRL | Qt.Key_Y),
TypeError: unsupported operand type(s) for |: 'Modifier' and 'Key'

@cbrnr Is this a bug in MNELAB? https://github.com/pyinstaller/pyinstaller/issues/7249#issuecomment-1316888347

[hoechenberger]

@larsoner Seems we've got more serious problems with the M1 build than I'd thought … it won't even resolve the deps if MNELAB is not included, it seems?

[hoechenberger]

Or is the arm64 selector perhaps not working 🤔

[mscheltienne]

I don't think spyder supports Qt6 either

It definitely does not. https://github.com/spyder-ide/spyder/blob/e5334d1c1e0b566147739b766d95127d145d0e3d/spyder/requirements.py#L37

[hoechenberger]

Or is the arm64 selector perhaps not working 🤔

I'm still not sure, but according to the build logs, we're trying to include pyside6 on ARM, which is bound to fail. Not sure where this is coming from though.

[hoechenberger]

Whoops. What's up now 🤯

[larsoner]

Looks like a conflict between Qt5 and Qt6

[hoechenberger]

Do we still need Qt5? If we drop Spyder, ...

[mscheltienne]

Would it be possible to maintain at least spyder-kernels? At least this way the environment can be quickly linked to an outside Spyder instance.

[hoechenberger]

There are tutorials out there where they embed Qt5 Matplotlib plots in a Qt6 application, so they should theoretically be able to live side-by-side…

[hoechenberger]

@larsoner

❯ mamba create -n test qt-main
❯ ls -lah ~/mambaforge/envs/test/plugins/platforms
total 6904
drwxr-xr-x   6 hoechenberger  staff   192B Dec 14 19:27 .
drwxr-xr-x  30 hoechenberger  staff   960B Dec 14 19:27 ..
-rwxrwxr-x   2 hoechenberger  staff   1.3M Dec 10 21:42 libqcocoa.dylib
-rwxrwxr-x   2 hoechenberger  staff   608K Dec 10 21:42 libqminimal.dylib
-rwxrwxr-x   2 hoechenberger  staff   637K Dec 10 21:41 libqoffscreen.dylib
-rwxrwxr-x   2 hoechenberger  staff   875K Dec 10 21:41 libqwebgl.dylib

but

❯ mamba create -n test qt6-main
❯ ls -lah ~/mambaforge/envs/test/plugins/platforms
ls: mambaforge/envs/test/plugins/platforms: No such file or directory

[larsoner]

We have a test that PyQt5 works for this repo. We should add an equivalent PySide6 test. It should fail based on what you write above I think...

[hoechenberger]

@larsoner

but

❯ mamba create -n test qt6-main
❯ ls -lah ~/mambaforge/envs/test/plugins/platforms
ls: mambaforge/envs/test/plugins/platforms: No such file or directory

Oh:

❯ ls -lah ~/mambaforge/envs/test/lib/qt6/plugins/platforms
total 2432
drwxr-xr-x   5 hoechenberger  staff   160B Dec 14 19:33 .
drwxr-xr-x  20 hoechenberger  staff   640B Dec 14 19:33 ..
-rwxrwxr-x   2 hoechenberger  staff   987K Oct 19 11:19 libqcocoa.dylib
-rwxrwxr-x   2 hoechenberger  staff    73K Oct 19 11:19 libqminimal.dylib
-rwxrwxr-x   2 hoechenberger  staff   150K Oct 19 11:19 libqoffscreen.dylib

So the plugins have moved. Maybe we need to set an environment variable or something…

[hoechenberger]

❯ mamba create -n test pyside6 pyqt matplotlib-base
❯ mamba run -n test python -c "import matplotlib; matplotlib.use('Qt5Agg')"

This works…

[hoechenberger]

Oh! Managed to reproduce.

❯ mamba create -n test pyside6 pyqt matplotlib-base
❯ mamba run -n test python -c "import matplotlib; import matplotlib.pyplot as plt; matplotlib.use('QtAgg'); plt.subplots()"
qt.core.plugin.loader: In /Users/hoechenberger/mambaforge/envs/test/plugins/platforms/libqwebgl.dylib:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.core.plugin.loader: In /Users/hoechenberger/mambaforge/envs/test/plugins/platforms/libqoffscreen.dylib:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.core.plugin.loader: In /Users/hoechenberger/mambaforge/envs/test/plugins/platforms/libqminimal.dylib:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.core.plugin.loader: In /Users/hoechenberger/mambaforge/envs/test/plugins/platforms/libqcocoa.dylib:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.qpa.plugin: Could not find the Qt platform plugin "cocoa" in ""
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

/var/folders/2k/nkw3dkx97ssdv08xmdgfnjkc0000gn/T/tmp92fve324: line 3: 89284 Abort trap: 6           python -c 'import matplotlib; import matplotlib.pyplot as plt; matplotlib.use('"'"'QtAgg'"'"'); plt.subplots()'

ERROR conda.cli.main_run:execute(49): `conda run python -c import matplotlib; import matplotlib.pyplot as plt; matplotlib.use('QtAgg'); plt.subplots()` failed. (See above for error)

But this works:

❯ mamba create -n test pyside6 pyqt matplotlib-base
❯ mamba run -n test python -c "import matplotlib; import matplotlib.pyplot as plt; matplotlib.use('Qt5Agg'); plt.subplots()"

tl;dr: Reuqesting the PyQt5 backend instead of the PyQt backend works.

[larsoner]

Pushed a commit with the suggested test I referred to before (check if pyside6 works at all by itself first)

[larsoner]

It appears that even just trying the basic command fails, even on Ubuntu:

$ LD_DEBUG=libs python -c "from PySide6.QtWidgets import QApplication, QWidget; app = QApplication([])"
...
      4288: calling init: /home/runner/mne-python/1.2.3_1/lib/python3.10/site-packages/PySide6/QtCore.abi3.so
      4288: 
qt.core.plugin.loader: In /home/runner/mne-python/1.2.3_1/plugins/platforms/libqvnc.so:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.core.plugin.loader: In /home/runner/mne-python/1.2.3_1/plugins/platforms/libqeglfs.so:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.core.plugin.loader: In /home/runner/mne-python/1.2.3_1/plugins/platforms/libqxcb.so:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.core.plugin.loader: In /home/runner/mne-python/1.2.3_1/plugins/platforms/libqwebgl.so:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.core.plugin.loader: In /home/runner/mne-python/1.2.3_1/plugins/platforms/libqminimalegl.so:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.core.plugin.loader: In /home/runner/mne-python/1.2.3_1/plugins/platforms/libqminimal.so:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.core.plugin.loader: In /home/runner/mne-python/1.2.3_1/plugins/platforms/libqoffscreen.so:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.qpa.plugin: Could not find the Qt platform plugin "xcb" in ""
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

/home/runner/work/_temp/408770a1-d657-4645-9c22-b10cbb7969a8.sh: line 43:  4288 Aborted                 (core dumped) LD_DEBUG=libs python -c "from PySide6.QtWidgets import QApplication, QWidget; app = QApplication([])"

https://github.com/mne-tools/mne-installers/actions/runs/3698421716/jobs/6265118735#step:13:1945

This is likely an untested case at the conda-forge end, and/or they don't support simultaneous Qt5/Qt6 installs, not sure...

[hoechenberger]

and/or they don't support simultaneous Qt5/Qt6 installs

If I'm not mistaken, I remember the person contributing the Qt6 packages spent a significant amount of time figuring out how to install both packages in parallel / how to isolate them. But I'm not sure if parallel install was actually extensively tested...

[hoechenberger]

@larsoner

And apparently, QT_PLUGIN_PATH is being ignored…

❯ QT_PLUGIN_PATH=/Users/hoechenberger/mambaforge/envs/test/lib/qt6/plugins python -c "from PySide6.QtWidgets import QApplication, QWidget; app = QApplication([])"
qt.core.plugin.loader: In /Users/hoechenberger/mambaforge/envs/test/plugins/platforms/libqwebgl.dylib:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.core.plugin.loader: In /Users/hoechenberger/mambaforge/envs/test/plugins/platforms/libqoffscreen.dylib:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.core.plugin.loader: In /Users/hoechenberger/mambaforge/envs/test/plugins/platforms/libqminimal.dylib:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.core.plugin.loader: In /Users/hoechenberger/mambaforge/envs/test/plugins/platforms/libqcocoa.dylib:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.core.plugin.loader: In /Users/hoechenberger/mambaforge/envs/test/plugins/platformthemes/libqxdgdesktopportal.dylib:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.core.plugin.loader: In /Users/hoechenberger/mambaforge/envs/test/plugins/styles/libqmacstyle.dylib:
  Plugin uses incompatible Qt library (5.15.0) [release]

[hoechenberger]

Setting QT_QPA_PLATFORM_PLUGIN_PATH doesn't seem to help either.

[hoechenberger]

@larsoner I'm hoping to generate more and more specific debugging output via b8d5ecc

[cbrnr]

I cannot run MNELAB on M1:

❯ mamba create -n test mnelab
❯ mamba activate test
❯ mnelab
...
TypeError: unsupported operand type(s) for |: 'Modifier' and 'Key'

@cbrnr Is this a bug in MNELAB? pyinstaller/pyinstaller#7249 (comment)

No, this is a breaking change from PySide6 < 6.4 vs. PySide6 ≥ 6.4. I adapted my code so that it works with the new types, but forgot to bump the dependency. I will do that right now!

Which version of PySide6 were you running when this problem happened?

[hoechenberger]

This was with PySide 6.4.1

[cbrnr]

This was with PySide 6.4.1

Are you sure? The code works with PySide6 ≥ 6.4, I just tested it with 6.4.0 and 6.4.1. The error message indicates that you were running PySide6 < 6.4, before they changed the types of some constants.

[hoechenberger]

Are you sure?

Yes:

❯ mamba create -y -n test pyside6=6.4.1 mnelab=0.8.5
❯ mamba run -n test mnelab
2022-12-15 11:10:34.021 python[21044:872306] ApplePersistenceIgnoreState: Existing state will not be touched. New state will be written to /var/folders/2k/nkw3dkx97ssdv08xmdgfnjkc0000gn/T/com.continuum.python.savedState
Traceback (most recent call last):
  File "/Users/hoechenberger/mambaforge/envs/test/bin/mnelab", line 11, in <module>
    sys.exit(main())
  File "/Users/hoechenberger/mambaforge/envs/test/lib/python3.10/site-packages/mnelab/__init__.py", line 42, in main
    model.view = MainWindow(model)
  File "/Users/hoechenberger/mambaforge/envs/test/lib/python3.10/site-packages/mnelab/mainwindow.py", line 285, in __init__
    QKeySequence(Qt.CTRL | Qt.Key_Y),
TypeError: unsupported operand type(s) for |: 'Modifier' and 'Key'

ERROR conda.cli.main_run:execute(49): `conda run mnelab` failed. (See above for error)

❯ mamba run -n test python -c "import PySide6; print(PySide6.__version__)"
6.4.1

[hoechenberger]

@cbrnr The problem does not occur with PySide6 6.4.1 installed via pip. Uuuughh....

[cbrnr]

The problem does not occur with PySide6 6.4.1 installed via pip. Uuuughh....

That's what I suspected. Do you know if they apply patches to the source? It looks like some custom configuration going on, which should probably be reported.

[hoechenberger]

I've opened an issue: https://github.com/conda-forge/pyside2-feedstock/issues/164