search

labstreaminglayer / pylsl

Python bindings (pylsl) for liblsl
MIT License
142 stars 58 forks source link

software architecture problem with pylsl & multiprocess.Process Python #58

Closed matpadalino87 closed 1 year ago

matpadalino87 commented 1 year ago

Dear all, I'd like to use LSL within a Python multiprocess.Process for reading in data from a stream with inlet.pull_chunk (and I am using a Windows machine Python 3.8). I'm working on Spyder Conda and I need to run 3/4 processes contemporary of which the first has the inlet object as argument for pulling the chunk in a while loop. After the creation of the inlet object in the main script, when the first process is started, I get the following exception however:

ctypes objects containing pointers cannot be pickled.

and later:

Traceback (most recent call last): File "", line 1, in File "C:\Users\Matteo\anaconda3\envs\ISO-CLS\lib\site-packages\multiprocess\spawn.py", line 116, in spawn_main exitcode = _main(fd, parent_sentinel) File "C:\Users\Matteo\anaconda3\envs\ISO-CLS\lib\site-packages\multiprocess\spawn.py", line 126, in _main self = reduction.pickle.load(from_parent) File "C:\Users\Matteo\anaconda3\envs\ISO-CLS\lib\site-packages\dill_dill.py", line 373, in load return Unpickler(file, ignore=ignore, **kwds).load() File "C:\Users\Matteo\anaconda3\envs\ISO-CLS\lib\site-packages\dill_dill.py", line 646, in load obj = StockUnpickler.load(self) EOFError: Ran out of input

If I use threading module there are no problems and the inlet object provides the requested of pulling chunk. I don't have the necessary skill to create and to develop a wrapper file to solve this problem, like have been suggested me, but I'd like to know if there is a smart solution.

If there is someone that can help me, will have my esteem and attention!

best regards

Matteo

cboulay commented 1 year ago

of which the first has the inlet object as argument

I take this to mean that you are creating the inlet in the primary process & primary thread, then you are passing the created inlet to a sub-process. You can't do this.

Instead, create the inlet in the sub-process. The primary process will have to pass information about how to find the stream (e.g., stream name, stream type) to the sub-process, but then the resolver and the inlet will run in the sub-process. This should be fine.

matpadalino87 commented 1 year ago

Thank you very much for your suggestion! Now the software totally runs without problems.