Closed larsoner closed 3 years ago
Okay looks like it's not a threading problem -- doing this with the built-in examples works fine:
$ python -u SendData.py & python -u ReceiveData.py
[1] 132179
looking for an EEG stream...
2021-02-08 15:27:11.392 ( 0.000s) [ 2CC42740] common.cpp:50 INFO| v1.14.0
now sending data...
2021-02-08 15:27:11.891 ( 0.500s) [ 7E6CC740] common.cpp:50 INFO| v1.14.0
1612816032.1011267 [0.32223236560821533, 0.4601549804210663, 0.6337152719497681, 0.37596964836120605, 0.6952264308929443, 0.7275350689888, 0.5970755815505981, 0.5940881967544556]
1612816032.1011553 [0.5168118476867676, 0.18368901312351227, 0.8172319531440735, 0.6709344387054443, 0.7501394152641296, 0.33982735872268677, 0.8810415863990784, 0.16764724254608154]
...
But if I add a import scipy.fft
to the ReceiveData.py script, it just hangs:
$ python -u SendData.py & python -u ReceiveData.py
[2] 131766
2021-02-08 15:25:36.148 ( 0.000s) [ 2FBBB740] common.cpp:50 INFO| v1.14.0
now sending data...
looking for an EEG stream...
So something about the pylsl and scipy.fft (and other modules) seems not to work.
I've tried this on latest scipy
master built from source, plus SciPy 1.6.0 from PyPi. And CircleCI using pip also has a variant of this problem on our CIs.
... and it looks like it works okay with pylsl-1.12.2
but not pylsl-1.13.1
based on pip install
-based version testing. So something about the code and/or lib change over that span seems to have broken things.
If it matters, this is all Python 3.9 on Ubuntu 20.10 x86_64.
What Linux distro are you on? I only have Ubuntu 20.04 in easy reach. Edit: Sorry I saw the Linux distro after.
It would help narrow down the problem tremendously if you could please try the following.
pip install pylsl
to get the latest version. Then git clone https://github.com/sccn/liblsl.git
, build that, and copy the liblsl.so it outputs to the pylsl folder (replacing the old liblsl.so). If this works then we know it's not a liblsl version problem, it's just how liblsl is built for the linux pylsl packages on pypi. (Aside: there seems to be a problem with the MacOS packages too for some versions)
We're hoping the next version of liblsl will have a different distribution process. See https://github.com/sccn/liblsl/issues/110 Then we don't need to build liblsl for manylinux, so hopefully we can avoid this problem altogether. Getting liblsl into Debian or NeuroDebian might take longer than you'd like, so maybe we can start with a custom conda channel.
is it expected that the conda-forge version will not have this problem?
Sorry what are you referring to here? Did you test something installed from conda and it didn't have the bug? I'm guessing not pylsl because that's not on conda-forge.
I 1) git clone
d this repo, 2) pip install -ve .
ed it to use latest master
, 3) git clone
d liblsl.git
, and then:
$ cmake -S . -B build -G Ninja
$ cd build
$ ninja -j 4
$ ln -s $PWD/liblsl.so ../../liblsl-Python/pylsl/lib/
And everything works! And when I replace liblsl-Python/pylsl/lib/liblsl.so
with:
~/.local/lib/python3.9/site-packages/pylsl/lib/liblsl.so
from the 1.14.0 installation by pip
- failsAnd then I verified that the 1.14.0 manylinux2010 library has the same MD5 (1793cbd2a45621572a14ff9090c57f32
) as the 1.14.0 from PyPi / ~/.local/...
. So it seems like the manylinux2010 build is for some reason problematic.
Thank you so much! This was tremendously helpful. I deleted all of the manylinux2010 releases from pypi. I hope the manylinux1 release is adequate for most users until we start shipping liblsl through package managers.
Indeed I just did pip uninstall pylsl; pip install --no-cache pylsl
and things appear to be working, thanks!
This bit of code is adapted from two pylsl examples and just starts one thread to send samples and receives them in the main thread, works on Linux (Ubuntu 20.10, pip) as it correctly starts printing a bunch of random samples:
However, if I move the
import scipy.fft
to before thepylsl
import, things break. The stream is never found and it just hangs on theresolve_stream
step. I narrowed thefrom scipy import fft
issue down to it being due to an import ofscipy.fft._pocketfft.pypocketfft
, which is a pybind11-compiled module. (I can also experience it when other modules are imported in that same spot in the script, so I don't think it's specific to SciPy, though.)This is on Linux, where I noticed there is a known issue that multiple threads are not supported. Is this what is meant by "not supported" in that it might work in some situations but not others depending on the import order? If so, is it expected that the conda-forge version will not have this problem, or is it some fundamental limitation? Also I'm not sure if this is related to #6 / #5 or not.
The above example is a minimized version of how we unit test and build docs for MNE-Realtime, hence my interest in it being fixed if at all possible.