pydoit / doit

CLI task management & automation tool
http://pydoit.org
MIT License
1.87k stars 175 forks source link

Program hangs when num process is set to 1 for using multiprocessing #461

Open sandeep-gh opened 1 year ago

sandeep-gh commented 1 year ago

For the following

DOIT_CONFIG = {
    "parallel_type": 'process',
    "num_process": 2,
}

the program hangs:

File "/home/adming/.local/bin/doit", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/adming/.local/lib/python3.11/site-packages/doit/__main__.py", line 8, in main
    sys.exit(DoitMain().run(sys.argv[1:]))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/adming/.local/lib/python3.11/site-packages/doit/doit_cmd.py", line 294, in run
    return command.parse_execute(args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/adming/.local/lib/python3.11/site-packages/doit/cmd_base.py", line 150, in parse_execute
    return self.execute(params, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/adming/.local/lib/python3.11/site-packages/doit/cmd_base.py", line 570, in execute
    return self._execute(**exec_params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/adming/.local/lib/python3.11/site-packages/doit/cmd_run.py", line 265, in _execute
    return runner.run_all(self.control.task_dispatcher())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/adming/.local/lib/python3.11/site-packages/doit/runner.py", line 254, in run_all
    self.run_tasks(task_dispatcher)
  File "/home/adming/.local/lib/python3.11/site-packages/doit/runner.py", line 463, in run_tasks
    result = result_q.get()
             ^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/multiprocessing/queues.py", line 103, in get
    res = self._recv_bytes()
          ^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/multiprocessing/connection.py", line 215, in recv_bytes
    buf = self._recv_bytes(maxlength)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/multiprocessing/connection.py", line 413, in _recv_bytes
    buf = self._recv(4)
          ^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/multiprocessing/connection.py", line 378, in _recv
    chunk = read(handle, remaining)
            ^^^^^^^^^^^^^^^^^^^^^^^
KeyboardInterrupt
^CException ignored in: <module 'threading' from '/usr/lib64/python3.11/threading.py'>

Environment

OS: Fedora 37 (6.5.5-100.fc37.aarch64)
python version: 3.11.5
doit version: VERSION = (0, 36, 0)

Upvote & Fund

Fund with Polar