Prerequisite

[X] I have searched Issues and Discussions but cannot get the expected help.
[X] I have read the FAQ documentation but cannot get the expected help.
[X] The bug has not been fixed in the latest version (master) or latest version (3.x).

Task

I'm using the official example scripts/configs for the officially supported tasks/models/datasets.

Branch

master branch https://github.com/open-mmlab/mmdetection

Environment

fatal: Not a git repository (or any of the parent directories): .git sys.platform: win32 Python: 3.8.0 (default, Nov 6 2019, 16:00:02) [MSC v.1916 64 bit (AMD64)] CUDA available: True GPU 0,1,2,3,4,5: Tesla P40 CUDA_HOME: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.3
NVCC: Cuda compilation tools, release 11.3, V11.3.58 MSVC: 用于 x64 的 Microsoft (R) C/C++ 优化编译器 19.16.27051 版 GCC: n/a PyTorch: 1.10.1+cu113 PyTorch compiling details: PyTorch built with:

C++ Version: 199711
MSVC 192829337
Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
Intel(R) MKL-DNN v2.2.3 (Git Hash 7336ca9f055cf1bfa13efb658fe15dc9b41f0740)
OpenMP 2019
LAPACK is enabled (usually provided by MKL)
CPU capability usage: AVX512
CUDA Runtime 11.3
NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
CuDNN 8.2
Magma 2.5.4
Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.2.0, CXX_COMPILER=C:/w/b/windows/tmp_bin/sccache-cl.exe, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /w /bigobj -DUSE_PTHREADPOOL -openmp:experimental -IC:/w/b/windows/mkl/include -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DUSE_FBGEMM -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.10.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON,

TorchVision: 0.11.2+cu113 OpenCV: 4.9.0 MMCV: 1.7.2 MMCV Compiler: MSVC 191627051 MMCV CUDA Compiler: 11.3 MMDetection: 2.26.0+

Reproduces the problem - code sample

def _update_iq_score_info(self, cat_id, cur_gt_roi_feat): cur_gt_roi_feat = cur_gt_roi_feat.view(-1, 256, 7, 7)

update the iq_score queue and corresponding dict info

    device_dir = str(cur_gt_roi_feat.device.index)
    cur_gt_save_pth = os.path.join(
        self.con_queue_dir, device_dir, str(cat_id) + '.pt')
    if os.path.exists(cur_gt_save_pth):
        # print("before:"+cur_gt_save_pth)
        cur_pt = torch.load(cur_gt_save_pth).view(-1, 256, 7, 7)
        os.remove(cur_gt_save_pth)
        # print("now:"+cur_gt_save_pth)
        cur_gt_roi_feat = torch.cat(
            [cur_pt.to(cur_gt_roi_feat.device), cur_gt_roi_feat], dim=0)
    cur_gt_roi_feat = cur_gt_roi_feat.view(-1, 256, 7, 7)
    dup_len = cur_gt_roi_feat.size(0) > int(self.num_con_queue // self.num_gpus)
    if dup_len > 0:
        cur_gt_roi_feat = cur_gt_roi_feat[-dup_len, ...]
    print("now:"+cur_gt_save_pth)
    torch.save(
        cur_gt_roi_feat, cur_gt_save_pth, _use_new_zipfile_serialization=False)

Reproduces the problem - command or script

python -m torch.distributed.launch --nproc_per_node=6 --master_port=29500 train.py --config ../configs/cfinet/faster_rcnn_r50_fpn_cfinet_1x.py --launcher pytorch

Reproduces the problem - error message

FileNotFoundError: [Errno 2] No such file or directory: './work_dirs/roi_feats/cfinet\1\5.pt'

Additional information

No response

shaunyuan22 / CFINet

[Bug] FileNotFoundError #26