gotcha / ipdb

Integration of IPython pdb
BSD 3-Clause "New" or "Revised" License
1.85k stars 146 forks source link

`import ipdb` stops multiprocessing from working #194

Open ppwwyyxx opened 4 years ago

ppwwyyxx commented 4 years ago
import ipdb
import multiprocessing as mp

def main():
    print(1)

if __name__ == "__main__":
    mp = mp.get_context("spawn")
    p = mp.Process(target=main, daemon=False)
    p.start()

gives:

Traceback (most recent call last):
  File "train_net.py", line 19, in <module>
    p.start()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/usr/lib/python3.8/multiprocessing/context.py", line 283, in _Popen
    return Popen(process_obj)
  File "/usr/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/usr/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/usr/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function main at 0x7f85d1b981f0>: attribute lookup main on __main__ failed

works after removing import ipdb.

RalphMao commented 1 month ago

Any solution to this, or alternative to ipdb?

ppwwyyxx commented 1 month ago

@RalphMao I usually use this to debug a new process:

import multiprocessing as mp
import os, sys

def main():
    print(1)
    if True:  # if this process needs debug
         os.dup2(0, sys.stdin.fileno())
         import IPython as IP; IP.embed()

if __name__ == "__main__":
    mp = mp.get_context("spawn")
    p = mp.Process(target=main, daemon=False)
    p.start()