'can only test a child process'

[DonMuv]

Hi, I encountered this error assert self._parent_pid == os.getpid(), 'can only test a child process' when I run train.py with the MSLS dataset

[Tobias-Fischer]

Could you please provide the full stack trace?

[DonMuv]

Exception ignored in: <function _MultiProcessingDataLoaderIter.del at 0x7f4faf115ca0>6 [00:00<?, ?it/s] Traceback (most recent call last): Exception ignored in: <function _MultiProcessingDataLoaderIter.del at 0x7f4faf115ca0> File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1324, in del Traceback (most recent call last): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1324, in del self._shutdown_workers() File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1316, in _shutdown_workers if w.is_alive(): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/multiprocessing/process.py", line 160, in is_alive assert self._parent_pid == os.getpid(), 'can only test a child process' AssertionError: can only test a child process self._shutdown_workers() File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1316, in _shutdown_workers if w.is_alive(): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/multiprocessing/process.py", line 160, in is_alive assert self._parent_pid == os.getpid(), 'can only test a child process' AssertionError: can only test a child process Exception ignored in: <function _MultiProcessingDataLoaderIter.del at 0x7f4faf115ca0> Traceback (most recent call last): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1324, in del Exception ignored in: <function _MultiProcessingDataLoaderIter.del at 0x7f4faf115ca0> Traceback (most recent call last): self._shutdown_workers() File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1316, in _shutdown_workers File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1324, in del Exception ignored in: <function _MultiProcessingDataLoaderIter.del at 0x7f4faf115ca0> if w.is_alive(): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/multiprocessing/process.py", line 160, in is_alive self._shutdown_workers() File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1316, in _shutdown_workers assert self._parent_pid == os.getpid(), 'can only test a child process' AssertionError: can only test a child process Traceback (most recent call last): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1324, in del if w.is_alive(): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/multiprocessing/process.py", line 160, in is_alive assert self._parent_pid == os.getpid(), 'can only test a child process' AssertionError: can only test a child process self._shutdown_workers() File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1316, in _shutdown_workers if w.is_alive(): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/multiprocessing/process.py", line 160, in is_alive assert self._parent_pid == os.getpid(), 'can only test a child process' AssertionError: can only test a child process Exception ignored in: <function _MultiProcessingDataLoaderIter.del at 0x7f4faf115ca0> Traceback (most recent call last): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1324, in del self._shutdown_workers() File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1316, in _shutdown_workers if w.is_alive(): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/multiprocessing/process.py", line 160, in is_alive assert self._parent_pid == os.getpid(), 'can only test a child process' AssertionError: can only test a child process Exception ignored in: <function _MultiProcessingDataLoaderIter.del at 0x7f4faf115ca0> Traceback (most recent call last): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1324, in del self._shutdown_workers() File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1316, in _shutdown_workers if w.is_alive(): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/multiprocessing/process.py", line 160, in is_alive assert self._parent_pid == os.getpid(), 'can only test a child process' AssertionError: can only test a child process Exception ignored in: <function _MultiProcessingDataLoaderIter.del at 0x7f4faf115ca0> Traceback (most recent call last): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1324, in del self._shutdown_workers() File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1316, in _shutdown_workers if w.is_alive(): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/multiprocessing/process.py", line 160, in is_alive assert self._parent_pid == os.getpid(), 'can only test a child process' AssertionError: can only test a child process Exception ignored in: <function _MultiProcessingDataLoaderIter.del at 0x7f4faf115ca0> Traceback (most recent call last): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1324, in del self._shutdown_workers() File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1316, in _shutdown_workers if w.is_alive(): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/multiprocessing/process.py", line 160, in is_alive assert self._parent_pid == os.getpid(), 'can only test a child process' AssertionError: can only test a child process Exception ignored in: <function _MultiProcessingDataLoaderIter.del at 0x7f4faf115ca0> Traceback (most recent call last): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1324, in del self._shutdown_workers() File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1316, in _shutdown_workers if w.is_alive(): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/multiprocessing/process.py", line 160, in is_alive assert self._parent_pid == os.getpid(), 'can only test a child process' AssertionError: can only test a child process Exception ignored in: <function _MultiProcessingDataLoaderIter.del at 0x7f4faf115ca0> Traceback (most recent call last): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1324, in del self._shutdown_workers() File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1316, in _shutdown_workers if w.is_alive(): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/multiprocessing/process.py", line 160, in is_alive assert self._parent_pid == os.getpid(), 'can only test a child process' AssertionError: can only test a child process Exception ignored in: <function _MultiProcessingDataLoaderIter.del at 0x7f4faf115ca0> Traceback (most recent call last): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1324, in del self._shutdown_workers() File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1316, in _shutdown_workers if w.is_alive(): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/multiprocessing/process.py", line 160, in is_alive assert self._parent_pid == os.getpid(), 'can only test a child process' AssertionError: can only test a child process Exception ignored in: <function _MultiProcessingDataLoaderIter.del at 0x7f4faf115ca0> Traceback (most recent call last): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1324, in del self._shutdown_workers() File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1316, in _shutdown_workers if w.is_alive(): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/multiprocessing/process.py", line 160, in is_alive assert self._parent_pid == os.getpid(), 'can only test a child process' AssertionError: can only test a child process Exception ignored in: <function _MultiProcessingDataLoaderIter.del at 0x7f4faf115ca0> Traceback (most recent call last): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1324, in del self._shutdown_workers() File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1316, in _shutdown_workers if w.is_alive(): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/multiprocessing/process.py", line 160, in is_alive assert self._parent_pid == os.getpid(), 'can only test a child process' AssertionError: can only test a child process Exception ignored in: <function _MultiProcessingDataLoaderIter.del at 0x7f4faf115ca0> Traceback (most recent call last): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1324, in del self._shutdown_workers() File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1316, in _shutdown_workers if w.is_alive(): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/multiprocessing/process.py", line 160, in is_alive assert self._parent_pid == os.getpid(), 'can only test a child process' AssertionError: can only test a child process

Searching for hard negatives... Allocated: 170.45 MB
Cached: 4.72 GB
Exception ignored in: <function _MultiProcessingDataLoaderIter.del at 0x7f4faf115ca0>6 [00:32<?, ?it/s] Traceback (most recent call last): | 0/123 [00:00<?, ?it/s] File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1324, in del self._shutdown_workers() File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1316, in _shutdown_workers if w.is_alive(): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/multiprocessing/process.py", line 160, in is_alive assert self._parent_pid == os.getpid(), 'can only test a child process' AssertionError: can only test a child process Exception ignored in: <function _MultiProcessingDataLoaderIter.del at 0x7f4faf115ca0> Traceback (most recent call last): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1324, in del self._shutdown_workers() File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1316, in _shutdown_workers if w.is_alive(): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/multiprocessing/process.py", line 160, in is_alive assert self._parent_pid == os.getpid(), 'can only test a child process' AssertionError: can only test a child process Exception ignored in: <function _MultiProcessingDataLoaderIter.del at 0x7f4faf115ca0> Traceback (most recent call last): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1324, in del self._shutdown_workers() File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1316, in _shutdown_workers if w.is_alive(): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/multiprocessing/process.py", line 160, in is_alive assert self._parent_pid == os.getpid(), 'can only test a child process' AssertionError: can only test a child process Exception ignored in: <function _MultiProcessingDataLoaderIter.del at 0x7f4faf115ca0> Traceback (most recent call last): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1324, in del self._shutdown_workers() File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1316, in _shutdown_workers if w.is_alive(): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/multiprocessing/process.py", line 160, in is_alive assert self._parent_pid == os.getpid(), 'can only test a child process' AssertionError: can only test a child process Exception ignored in: <function _MultiProcessingDataLoaderIter.del at 0x7f4faf115ca0> Traceback (most recent call last): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1324, in del self._shutdown_workers() File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1316, in _shutdown_workers if w.is_alive(): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/multiprocessing/process.py", line 160, in is_alive assert self._parent_pid == os.getpid(), 'can only test a child process' AssertionError: can only test a child process Exception ignored in: <function _MultiProcessingDataLoaderIter.del at 0x7f4faf115ca0> Traceback (most recent call last): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1324, in del self._shutdown_workers() File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1316, in _shutdown_workers if w.is_alive(): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/multiprocessing/process.py", line 160, in is_alive assert self._parent_pid == os.getpid(), 'can only test a child process' AssertionError: can only test a child process Exception ignored in: <function _MultiProcessingDataLoaderIter.del at 0x7f4faf115ca0> Traceback (most recent call last): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1324, in del self._shutdown_workers() File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1316, in _shutdown_workers if w.is_alive(): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/multiprocessing/process.py", line 160, in is_alive assert self._parent_pid == os.getpid(), 'can only test a child process' AssertionError: can only test a child process Exception ignored in: <function _MultiProcessingDataLoaderIter.del at 0x7f4faf115ca0> Traceback (most recent call last): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1324, in del self._shutdown_workers() File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1316, in _shutdown_workers if w.is_alive(): File "/home/donmuv/anaconda3/envs/NetVlad/lib/python3.8/multiprocessing/process.py", line 160, in is_alive assert self._parent_pid == os.getpid(), 'can only test a child process'

[whnwwwhhhnnn]

May I ask how you resolved this issue？

[whu-lyh]

@Tobias-Fischer , @DonMuv , @whnwwwhhhnnn . I also met this error when I set the num_worker > 0, the training is stuck, and this error occurs:

Exception ignored in: <function _MultiProcessingDataLoaderIter.__del__ at 0x7f81c5877c20>
Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1328, in __del__
    self._shutdown_workers()
  File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1320, in _shutdown_workers
    if w.is_alive():
  File "/opt/conda/lib/python3.7/multiprocessing/process.py", line 151, in is_alive
    assert self._parent_pid == os.getpid(), 'can only test a child process'
AssertionError: can only test a child process
Exception ignored in: <function _MultiProcessingDataLoaderIter.__del__ at 0x7f81c5877c20>
Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1328, in __del__
    self._shutdown_workers()
  File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1320, in _shutdown_workers
    if w.is_alive():
  File "/opt/conda/lib/python3.7/multiprocessing/process.py", line 151, in is_alive
    assert self._parent_pid == os.getpid(), 'can only test a child process'

The program succeeds when the num_works==0, however, the training process is slow, and the GPU usage rate is also low. Any ideas to solve this error?

[whnwwwhhhnnn]

I set (persistent_workers=True) in (train_loader, val_loader) respectively, and the problem was solved.

------------------ 原始邮件 ------------------ 发件人: "QVPR/Patch-NetVLAD" @.>; 发送时间: 2023年8月6日(星期天) 上午9:42 @.>; @.**@.>; 主题: Re: [QVPR/Patch-NetVLAD] 'can only test a child process' (#34)

@Tobias-Fischer , @DonMuv , @whnwwwhhhnnn . I also met this error when I set the num_worker > 0, the training is stuck, and this error occurs: Exception ignored in: <function _MultiProcessingDataLoaderIter.del at 0x7f81c5877c20> Traceback (most recent call last): File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1328, in del self._shutdown_workers() File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1320, in _shutdown_workers if w.is_alive(): File "/opt/conda/lib/python3.7/multiprocessing/process.py", line 151, in is_alive assert self._parent_pid == os.getpid(), 'can only test a child process' AssertionError: can only test a child process Exception ignored in: <function _MultiProcessingDataLoaderIter.del at 0x7f81c5877c20> Traceback (most recent call last): File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1328, in del self._shutdown_workers() File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1320, in _shutdown_workers if w.is_alive(): File "/opt/conda/lib/python3.7/multiprocessing/process.py", line 151, in is_alive assert self._parent_pid == os.getpid(), 'can only test a child process'
The program succeeds when the num_works==0, however, the training process is slow, and the GPU usage rate is also low. Any ideas to solve this error?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

[whu-lyh]

Well, thanks!

[Tobias-Fischer]

Hi @whu-lyh @whnwwwhhhnnn - could you please open a pull request so we can fix this here? I am not sure which exact DataLoader you are referring to.