I'm getting a pickle error while using multiprocessing with Pyconcrete. However the code works just fine without pyconcrete.
I am using python 3.8 with docker. To create a docker image I am using the docker file from the repository https://github.com/Falldog/pyconcrete/blob/master/docker/Dockerfile.
The error I am getting is
Traceback (most recent call last): File "main.py", line 17, in <module> File "/usr/lib/python3.8/multiprocessing/process.py", line 121, in start self._popen = self._Popen(self) File "/usr/lib/python3.8/multiprocessing/context.py", line 284, in _Popen return Popen(process_obj) File "/usr/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__ super().__init__(process_obj) File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__ self._launch(process_obj) File "/usr/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch reduction.dump(process_obj, fp) File "/usr/lib/python3.8/multiprocessing/reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) _pickle.PicklingError: Can't pickle <function task_fn at 0x7f376688b1f0>: attribute lookup task_fn on __main__ failed
I also attach the code below:
import os
import time
def task_fn(l):
results = []
for each in l:
print(each**20)
if __name__ == '__main__':
print("Main process id ", os.getpid())
l = [i**2 for i in range(10)]
ctx = mp.get_context('spawn')
p = ctx.Process(target= task_fn, args = (l,))
p.start()
print(p.name)
p.join()
p.close()
Hi,
I'm getting a pickle error while using multiprocessing with Pyconcrete. However the code works just fine without pyconcrete. I am using python 3.8 with docker. To create a docker image I am using the docker file from the repository https://github.com/Falldog/pyconcrete/blob/master/docker/Dockerfile.
The error I am getting is
Traceback (most recent call last): File "main.py", line 17, in <module> File "/usr/lib/python3.8/multiprocessing/process.py", line 121, in start self._popen = self._Popen(self) File "/usr/lib/python3.8/multiprocessing/context.py", line 284, in _Popen return Popen(process_obj) File "/usr/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__ super().__init__(process_obj) File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__ self._launch(process_obj) File "/usr/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch reduction.dump(process_obj, fp) File "/usr/lib/python3.8/multiprocessing/reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) _pickle.PicklingError: Can't pickle <function task_fn at 0x7f376688b1f0>: attribute lookup task_fn on __main__ failed
I also attach the code below: