search

facebookresearch / detectron2

Detectron2 is a platform for object detection, segmentation and other visual recognition tasks.
https://detectron2.readthedocs.io/en/latest/
Apache License 2.0
30.7k stars 7.51k forks source link

FileNotFoundError: [Errno 2] No such file or directory: 'datasets/coco/thing_train2017/000000009725.npz' #2627

Closed KHKHG closed 3 years ago

KHKHG commented 3 years ago
  1. What exact command you run: OMP_NUM_THREADS=1 python tools/train_net.py --config-file configs/BlendMask/Base-BlendMask.yaml --num-gpus 1 OUTPUT_DIR training_dir/Base-BlendMask
  2. Full logs or other relevant observations: 
    
Starting training from iteration 0
/root/AdelaiDet/detectron2/detectron2/structures/masks.py:348: UserWarning: This overload of nonzero is deprecated:
    nonzero()
Consider using one of the following signatures instead:
    nonzero(*, bool as_tuple) (Triggered internally at  /opt/conda/conda-bld/pytorch_1595629427478/work/torch/csrc/utils/python_arg_parser.cpp:766.)
item = item.nonzero().squeeze(1).cpu().numpy().tolist()
Traceback (most recent call last):
File "tools/train_net.py", line 213, in <module>
args=(args,),
File "/root/AdelaiDet/detectron2/detectron2/engine/launch.py", line 62, in launch
main_func(*args)
File "tools/train_net.py", line 201, in main
return trainer.train()
File "tools/train_net.py", line 84, in train
self.train_loop(self.start_iter, self.max_iter)
File "tools/train_net.py", line 74, in train_loop
self.run_step()
File "/root/AdelaiDet/detectron2/detectron2/engine/defaults.py", line 441, in run_step
self._trainer.run_step()
File "/root/AdelaiDet/detectron2/detectron2/engine/train_loop.py", line 226, in run_step
data = next(self._data_loader_iter)
File "/root/AdelaiDet/detectron2/detectron2/data/common.py", line 179, in __iter__
for d in self.dataset:
File "/usr/local/miniconda3/envs/ADNET/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 363, in __next__
data = self._next_data()
File "/usr/local/miniconda3/envs/ADNET/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 989, in _next_data
return self._process_data(data)
File "/usr/local/miniconda3/envs/ADNET/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1014, in _process_data
data.reraise()
File "/usr/local/miniconda3/envs/ADNET/lib/python3.7/site-packages/torch/_utils.py", line 395, in reraise
raise self.exc_type(msg)
FileNotFoundError: Caught FileNotFoundError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/usr/local/miniconda3/envs/ADNET/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 185, in _worker_loop
data = fetcher.fetch(index)
File "/usr/local/miniconda3/envs/ADNET/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/usr/local/miniconda3/envs/ADNET/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/root/AdelaiDet/detectron2/detectron2/data/common.py", line 43, in __getitem__
data = self._map_func(self._dataset[cur_idx])
File "/root/AdelaiDet/detectron2/detectron2/utils/serialize.py", line 23, in __call__
return self._obj(*args, **kwargs)
File "/root/AdelaiDet/adet/data/dataset_mapper.py", line 207, in __call__
basis_sem_gt = np.load(basis_sem_path)["mask"]
File "/usr/local/miniconda3/envs/ADNET/lib/python3.7/site-packages/numpy/lib/npyio.py", line 416, in load
fid = stack.enter_context(open(os_fspath(file), "rb"))
FileNotFoundError: [Errno 2] No such file or directory: 'datasets/coco/thing_train2017/000000009725.npz'

/root/AdelaiDet/detectron2/detectron2/structures/masks.py:348: UserWarning: This overload of nonzero is deprecated: nonzero() Consider using one of the following signatures instead: nonzero(, bool as_tuple) (Triggered internally at /opt/conda/conda-bld/pytorch_1595629427478/work/torch/csrc/utils/python_arg_parser.cpp:766.) item = item.nonzero().squeeze(1).cpu().numpy().tolist() /root/AdelaiDet/detectron2/detectron2/structures/masks.py:348: UserWarning: This overload of nonzero is deprecated: nonzero() Consider using one of the following signatures instead: nonzero(, bool as_tuple) (Triggered internally at /opt/conda/conda-bld/pytorch_1595629427478/work/torch/csrc/utils/python_arg_parser.cpp:766.) item = item.nonzero().squeeze(1).cpu().numpy().tolist()


3. please simplify the steps as much as possible so they do not require additional resources to
     run, such as a private dataset.
I just use coco2017:
wget -c http://images.cocodataset.org/zips/train2017.zip
wget -c http://images.cocodataset.org/annotations/annotations_trainval2017.zip
wget -c http://images.cocodataset.org/zips/test2017.zip
wget -c http://images.cocodataset.org/annotations/stuff_annotations_trainval2017.zip
wget -c http://images.cocodataset.org/annotations/image_info_test2017.zip

and change like this:
 https://github.com/aim-uofa/AdelaiDet/blob/master/tools/train_net.py

## Environment:
pytorch==1.6.0 torchvision==0.7.0 cudatoolkit=10.2 python=3.7
github-actions[bot] commented 3 years ago

You've chosen to report an unexpected problem or bug. Unless you already know the root cause of it, please include details about it by filling the issue template. The following information is missing: "Instructions To Reproduce the Issue and Full Logs";

KHKHG commented 3 years ago

FULL LOGS:

[02/15 13:28:55 d2.data.build]: Removed 1021 images with no usable annotations. 117266 images left. [02/15 13:29:03 d2.data.build]: Distribution of instances among all 80 categories: category #instances category #instances category #instances
person 257253 bicycle 7056 car 43533
motorcycle 8654 airplane 5129 bus 6061
train 4570 truck 9970 boat 10576
traffic light 12842 fire hydrant 1865 stop sign 1983
parking meter 1283 bench 9820 bird 10542
cat 4766 dog 5500 horse 6567
sheep 9223 cow 8014 elephant 5484
bear 1294 zebra 5269 giraffe 5128
backpack 8714 umbrella 11265 handbag 12342
tie 6448 suitcase 6112 frisbee 2681
skis 6623 snowboard 2681 sports ball 6299
kite 8802 baseball bat 3273 baseball gl.. 3747
skateboard 5536 surfboard 6095 tennis racket 4807
bottle 24070 wine glass 7839 cup 20574
fork 5474 knife 7760 spoon 6159
bowl 14323 banana 9195 apple 5776
sandwich 4356 orange 6302 broccoli 7261
carrot 7758 hot dog 2884 pizza 5807
donut 7005 cake 6296 chair 38073
couch 5779 potted plant 8631 bed 4192
dining table 15695 toilet 4149 tv 5803
laptop 4960 mouse 2261 remote 5700
keyboard 2854 cell phone 6422 microwave 1672
oven 3334 toaster 225 sink 5609
refrigerator 2634 book 24077 clock 6320
vase 6577 scissors 1464 teddy bear 4729
hair drier 198 toothbrush 1945
total 849949

[02/15 13:29:03 d2.data.build]: Using training sampler TrainingSampler [02/15 13:29:03 d2.data.common]: Serializing 117266 elements to byte tensors and concatenating them all ... [02/15 13:29:09 d2.data.common]: Serialized dataset takes 451.21 MiB [02/15 13:29:14 fvcore.common.checkpoint]: No checkpoint found. Initializing model from scratch [02/15 13:29:14 adet.trainer]: Starting training from iteration 0 /root/AdelaiDet/detectron2/detectron2/structures/masks.py:348: UserWarning: This overload of nonzero is deprecated: nonzero() Consider using one of the following signatures instead: nonzero(, bool as_tuple) (Triggered internally at /opt/conda/conda-bld/pytorch_1595629427478/work/torch/csrc/utils/python_arg_parser.cpp:766.) item = item.nonzero().squeeze(1).cpu().numpy().tolist() Traceback (most recent call last): File "tools/train_net.py", line 213, in args=(args,), File "/root/AdelaiDet/detectron2/detectron2/engine/launch.py", line 62, in launch main_func(args) File "tools/train_net.py", line 201, in main return trainer.train() File "tools/train_net.py", line 84, in train self.train_loop(self.start_iter, self.max_iter) File "tools/train_net.py", line 74, in train_loop self.run_step() File "/root/AdelaiDet/detectron2/detectron2/engine/defaults.py", line 441, in run_step self._trainer.run_step() File "/root/AdelaiDet/detectron2/detectron2/engine/train_loop.py", line 226, in run_step data = next(self._data_loader_iter) File "/root/AdelaiDet/detectron2/detectron2/data/common.py", line 179, in iter for d in self.dataset: File "/usr/local/miniconda3/envs/ADNET/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 363, in next data = self._next_data() File "/usr/local/miniconda3/envs/ADNET/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 989, in _next_data return self._process_data(data) File "/usr/local/miniconda3/envs/ADNET/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1014, in _process_data data.reraise() File "/usr/local/miniconda3/envs/ADNET/lib/python3.7/site-packages/torch/_utils.py", line 395, in reraise raise self.exc_type(msg) FileNotFoundError: Caught FileNotFoundError in DataLoader worker process 0. Original Traceback (most recent call last): File "/usr/local/miniconda3/envs/ADNET/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 185, in _worker_loop data = fetcher.fetch(index) File "/usr/local/miniconda3/envs/ADNET/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/usr/local/miniconda3/envs/ADNET/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/root/AdelaiDet/detectron2/detectron2/data/common.py", line 43, in getitem data = self._map_func(self._dataset[cur_idx]) File "/root/AdelaiDet/detectron2/detectron2/utils/serialize.py", line 23, in call return self._obj(*args, **kwargs) File "/root/AdelaiDet/adet/data/dataset_mapper.py", line 207, in call basis_sem_gt = np.load(basis_sem_path)["mask"] File "/usr/local/miniconda3/envs/ADNET/lib/python3.7/site-packages/numpy/lib/npyio.py", line 416, in load fid = stack.enter_context(open(os_fspath(file), "rb")) FileNotFoundError: [Errno 2] No such file or directory: 'datasets/coco/thing_train2017/000000009725.npz'

/root/AdelaiDet/detectron2/detectron2/structures/masks.py:348: UserWarning: This overload of nonzero is deprecated: nonzero() Consider using one of the following signatures instead: nonzero(, bool as_tuple) (Triggered internally at /opt/conda/conda-bld/pytorch_1595629427478/work/torch/csrc/utils/python_arg_parser.cpp:766.) item = item.nonzero().squeeze(1).cpu().numpy().tolist() /root/AdelaiDet/detectron2/detectron2/structures/masks.py:348: UserWarning: This overload of nonzero is deprecated: nonzero() Consider using one of the following signatures instead: nonzero(, bool as_tuple) (Triggered internally at /opt/conda/conda-bld/pytorch_1595629427478/work/torch/csrc/utils/python_arg_parser.cpp:766.) item = item.nonzero().squeeze(1).cpu().numpy().tolist()

ppwwyyxx commented 3 years ago

File "/root/AdelaiDet/adet/data/dataset_mapper.py", line 207, in call basis_sem_gt = np.load(basis_sem_path)["mask"] File "/usr/local/miniconda3/envs/ADNET/lib/python3.7/site-packages/numpy/lib/npyio.py", line 416, in load fid = stack.enter_context(open(os_fspath(file), "rb")) FileNotFoundError: [Errno 2] No such file or directory: 'datasets/coco/thing_train2017/000000009725.npz'

The code that loads the file does not come from this project.

LiWentomng commented 3 years ago

The .npz file needs to be generated by the script prepare_thing_sem_from_instance.py for training BlendMask model. Please refer to this page.