Jingkang50 / OpenPSG

Benchmarking Panoptic Scene Graph Generation (PSG), ECCV'22
https://psgdataset.org
MIT License
407 stars 68 forks source link

Bad file descriptor Error after finishing the training process #99

Open Yassin-fan opened 1 year ago

Yassin-fan commented 1 year ago

Thanks for your work! I have tried to run this PSGTR on 3090 for 60 epochs. I didn't change the code, but when it finished the training process, there is an error:

2023-05-03 21:09:11,872 - mmdet - INFO - Epoch(val) [60][2177] * wandb: Waiting for W&B process to finish... (success). .... wandb: 🚀 View run psgtr_r50_psg_0.5_scale_mask at: https://wandb.ai/mygraduation/psgformer/runs/r** wandb: Synced 6 W&B file(s), 0 media file(s), 10 artifact file(s) and 0 other file(s) wandb: Find logs at: ./wandb/run-20230417_071322-r99zn89q/logs The program finished and will be restarted

/dockerFile/OpenPSG/tools/train.py(2)() -> import argparse (Pdb) Traceback (most recent call last): File "/opt/conda/envs/ymf_openpsg/lib/python3.7/pdb.py", line 1699, in main pdb._runscript(mainpyfile) File "/opt/conda/envs/ymf_openpsg/lib/python3.7/pdb.py", line 1568, in _runscript self.run(statement) File "/opt/conda/envs/ymf_openpsg/lib/python3.7/bdb.py", line 578, in run exec(cmd, globals, locals) File "", line 1, in File "/dockerFile/OpenPSG/tools/train.py", line 2, in import argparse File "/dockerFile/OpenPSG/tools/train.py", line 2, in import argparse File "/opt/conda/envs/ymf_openpsg/lib/python3.7/bdb.py", line 88, in trace_dispatch return self.dispatch_line(frame) File "/opt/conda/envs/ymf_openpsg/lib/python3.7/bdb.py", line 112, in dispatch_line self.user_line(frame) File "/opt/conda/envs/ymf_openpsg/lib/python3.7/pdb.py", line 259, in user_line self.interaction(frame, None) File "/opt/conda/envs/ymf_openpsg/lib/python3.7/pdb.py", line 350, in interaction self._cmdloop() File "/opt/conda/envs/ymf_openpsg/lib/python3.7/pdb.py", line 319, in _cmdloop self.cmdloop() File "/opt/conda/envs/ymf_openpsg/lib/python3.7/cmd.py", line 126, in cmdloop line = input(self.prompt) OSError: [Errno 9] Bad file descriptor Uncaught exception. Entering post mortem debugging Running 'cont' or 'step' will restart the program /opt/conda/envs/ymf_openpsg/lib/python3.7/cmd.py(126)cmdloop() -> line = input(self.prompt) (Pdb) Traceback (most recent call last): File "/opt/conda/envs/ymf_openpsg/lib/python3.7/pdb.py", line 1699, in main pdb._runscript(mainpyfile) File "/opt/conda/envs/ymf_openpsg/lib/python3.7/pdb.py", line 1568, in _runscript self.run(statement) File "/opt/conda/envs/ymf_openpsg/lib/python3.7/bdb.py", line 578, in run exec(cmd, globals, locals) File "", line 1, in File "/dockerFile/OpenPSG/tools/train.py", line 2, in import argparse File "/dockerFile/OpenPSG/tools/train.py", line 2, in import argparse File "/opt/conda/envs/ymf_openpsg/lib/python3.7/bdb.py", line 88, in trace_dispatch return self.dispatch_line(frame) File "/opt/conda/envs/ymf_openpsg/lib/python3.7/bdb.py", line 112, in dispatch_line self.user_line(frame) File "/opt/conda/envs/ymf_openpsg/lib/python3.7/pdb.py", line 259, in user_line self.interaction(frame, None) File "/opt/conda/envs/ymf_openpsg/lib/python3.7/pdb.py", line 350, in interaction self._cmdloop() File "/opt/conda/envs/ymf_openpsg/lib/python3.7/pdb.py", line 319, in _cmdloop self.cmdloop() File "/opt/conda/envs/ymf_openpsg/lib/python3.7/cmd.py", line 126, in cmdloop line = input(self.prompt) OSError: [Errno 9] Bad file descriptor

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/opt/conda/envs/ymf_openpsg/lib/python3.7/runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "/opt/conda/envs/ymf_openpsg/lib/python3.7/runpy.py", line 85, in _run_code exec(code, run_globals) File "/opt/conda/envs/ymf_openpsg/lib/python3.7/pdb.py", line 1726, in pdb.main() File "/opt/conda/envs/ymf_openpsg/lib/python3.7/pdb.py", line 1718, in main pdb.interaction(None, t) File "/opt/conda/envs/ymf_openpsg/lib/python3.7/pdb.py", line 350, in interaction self._cmdloop() File "/opt/conda/envs/ymf_openpsg/lib/python3.7/pdb.py", line 319, in _cmdloop self.cmdloop() File "/opt/conda/envs/ymf_openpsg/lib/python3.7/cmd.py", line 126, in cmdloop line = input(self.prompt) OSError: [Errno 9] Bad file descriptor

I searched this error and found that it is because the code somewhere has closed the connection prematurely? Is there something wrong? Thanks for your help.