"Can't pickle" functions in DEVICES and PermissionError: [WinError 5] Access is denied

[juliomateoslangerak]

When I use a function to customize a device in the config I get a nasty error.

Traceback (most recent call last):
  File "C:\Users\omxt\Applications\WPy64-3890\python-3.8.9.amd64\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\omxt\Applications\WPy64-3890\python-3.8.9.amd64\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\omxt\PycharmProjects\microscope\microscope\device_server.py", line 634, in <module>
    sys.exit(main(sys.argv))
  File "C:\Users\omxt\PycharmProjects\microscope\microscope\device_server.py", line 610, in main
    serve_devices(devices, options)
  File "C:\Users\omxt\PycharmProjects\microscope\microscope\device_server.py", line 482, in serve_devices
    servers[-1].start()
  File "C:\Users\omxt\Applications\WPy64-3890\python-3.8.9.amd64\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "C:\Users\omxt\Applications\WPy64-3890\python-3.8.9.amd64\lib\multiprocessing\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\omxt\Applications\WPy64-3890\python-3.8.9.amd64\lib\multiprocessing\context.py", line 327, in _Popen
    return Popen(process_obj)
  File "C:\Users\omxt\Applications\WPy64-3890\python-3.8.9.amd64\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Users\omxt\Applications\WPy64-3890\python-3.8.9.amd64\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function construct_camera at 0x0000000003C73DC0>: import of module 'config' failed

(microscope_venv38) C:\Users\omxt\PycharmProjects\microscope>Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\omxt\Applications\WPy64-3890\python-3.8.9.amd64\lib\multiprocessing\spawn.py", line 107, in spawn_main
    new_handle = reduction.duplicate(pipe_handle,
  File "C:\Users\omxt\Applications\WPy64-3890\python-3.8.9.amd64\lib\multiprocessing\reduction.py", line 79, in duplicate
    return _winapi.DuplicateHandle(
PermissionError: [WinError 5] Access is denied

This is reproducible with the code in the doc example (I removed the typehints). Use of 3.8 on Windows7

from microscope.device_server import device
from microscope.simulators import SimulatedCamera

def construct_camera():
    camera = SimulatedCamera()
    camera.set_setting("display image number", False)
    return {"DummyCamera": camera}

DEVICES = [
    device(construct_camera, host="127.0.0.1", port=8000),
]

Passing DEVICES = [device(SimulatedCamera, host="127.0.0.1", port=8001)] works as expected

[carandraug]

Seems that for some reason it's failing to pickle the function. This works in Python 3.7 and 3.9 in Linux. You're using Python 3.8 on Windows. Did this work with Python 3.7 in Windows? Or can you try other Python version?

[juliomateoslangerak]

Same thing in Windows 10 with python 3.9

[carandraug]

This is a duplicate of issue #179 . There's a proposed fix but there's some issues with it. There's a proposed plan to fix it but no one has implemented it yet. Closing as duplicate.

[juliomateoslangerak]

After reading this I hacked to add the function definition (and the DEVICES list) on the top level module (device_server.py) and that seems to work (for finding the problem I mean) From what I read in other places, the root reason seems to be windows not forking the process but spawning it.

[juliomateoslangerak]

Just saw tour comment. I'm closing this as duplicate in #179