Closed priyanka-chaudhary closed 6 years ago
@priyanka-chaudhary, weren't the 2014 versions of instances_valminusminival2014.json
and instances_minival2014.json
build by Ross Girshick and his team? That's the feeling I get from reading this README.md. I don't believe there's ever been an official release of these files.
May I suggest that you build your own versions for the 2017 dataset? If you go by the official COCO page, there is no additional image data but only new stuff annotations and new data splits:
2017 Update: The main change in 2017 is that instead of an 80K/40K train/val split, based on community feedback the split is now 115K/5K for train/val. The same exact images are used, and no new annotations for detection/keypoints are provided. However, new in 2017 are stuff annotations on 40K train images (subset of the full 115K train images from 2017) and 5K val images. Also, for testing, in 2017 the test set only has two splits (dev / challenge), instead of the four splits (dev / standard / reserve / challenge) used in previous years. Finally, new in 2017 we are releasing 120K unlabeled images from COCO that follow the same class distribution as the labeled images; this may be useful for semi-supervised learning on COCO.
Note: Annotations last updated 09/05/2017 (stuff annotations added). If you find any issues with the data please let us know!
Does this mean that instances_valminusminival2014.json
and instances_minival2014.json
files can be used with the 2017 dataset?
@philferriere, Is there somewhere instructions on how to build these two( instances_valminusminival2017.json and instances_minival2014.json) files then?
Also, then I assume people here have used 2014 version of these files here only?
@priyanka-chaudhary, since the data is the same, I'm not sure building a different version (instead of using the 2014 files) is actually meaningful.
For me, I changed 'instance_val2017.json' to 'instance_minival2017.json' and now it works.
@philferriere : Data overall combined(train + validation) is same. But validatation dataset of 2014 and 2017 are different also train dataset. That's why I am asking.
@LifeBeyondExpectations : Thank you. Also did you change something else too? As I am getting FileNotFoundError: [Errno 2] No such file or directory: '/home/pchaudha/mask-rcnn/Mask_RCNN-master/dataset-coco/val2017/COCO_val2014_000000130437.jpg'
So it is basically trying to find 2014 coco dataset files in 2017 folder. So some kind of list of images that also needs to be changed?
Also what did you do for file instances_valminusminival2017.json? Thanks again.
@priyanka-chaudhary did you find an answer to your problem? I'm keeps running into No file found issue. It's basically one after another. Now I'm receiving FileNotFoundError: [Errno 2] No such file or directory: '/ml/coco/data/val2017/000000372819.jpg'. Any ideas on this?
@aemilcar : The change 'instance_val2017.json' to 'instance_minival2017.json' and commenting line 477 in coco.py
dataset_train.load_coco(args.dataset, "valminusminival", year=args.year, auto_download=args.download)
After that it works for me.
I had to change it to instances_minival2014.json
not instance_minival2017.json
.
@CMCDragonkai but how did you get the instance_minival2017.json
I think I renamed the files, I didn't download anything new.
I am trying to train balloon.py file on balloon dataset. Both the train and test datasets contain via_region_data.json file. However the error that I am getting is "via_region_data.json" file not available. Trying to run in google colab with the below command. !python balloon.py train --dataset=Mask_RCNN-master/data/balloon --weights=coco
The error that I am getting is as below:
Using TensorFlow backend. Weights: coco Dataset: Mask_RCNN-master/dataset Logs: /content/drive/My Drive/Mask-rcnn1/logs
Configurations: BACKBONE resnet101 BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 2 BBOX_STD_DEV [0.1 0.1 0.2 0.2] COMPUTE_BACKBONE_SHAPE None DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.9 DETECTION_NMS_THRESHOLD 0.3 FPN_CLASSIF_FC_LAYERS_SIZE 1024 GPU_COUNT 1 GRADIENT_CLIP_NORM 5.0 IMAGES_PER_GPU 2 IMAGE_CHANNEL_COUNT 3 IMAGE_MAX_DIM 1024 IMAGE_META_SIZE 14 IMAGE_MIN_DIM 800 IMAGE_MIN_SCALE 0 IMAGE_RESIZE_MODE square IMAGE_SHAPE [1024 1024 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME balloon NUM_CLASSES 2 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 PRE_NMS_LIMIT 6000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (32, 64, 128, 256, 512) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 100 TOP_DOWN_PYRAMID_SIZE 256 TRAIN_BN False TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:541: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:66: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:4432: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:2139: The name tf.nn.fused_batch_norm is deprecated. Please use tf.compat.v1.nn.fused_batch_norm instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:4267: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:2239: The name tf.image.resize_nearest_neighbor is deprecated. Please use tf.compat.v1.image.resize_nearest_neighbor instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/array_ops.py:1475: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.where in 2.0, which has the same broadcast rule as np.where WARNING:tensorflow:From /content/drive/My Drive/Mask-rcnn1/Mask_RCNN-master/balloon/mrcnn/model.py:553: The name tf.random_shuffle is deprecated. Please use tf.random.shuffle instead.
WARNING:tensorflow:From /content/drive/My Drive/Mask-rcnn1/Mask_RCNN-master/balloon/mrcnn/utils.py:202: The name tf.log is deprecated. Please use tf.math.log instead.
WARNING:tensorflow:From /content/drive/My Drive/Mask-rcnn1/Mask_RCNN-master/balloon/mrcnn/model.py:600: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead Downloading pretrained model to /content/drive/My Drive/Mask-rcnn1/mask_rcnn_coco.h5 ... ... done downloading pretrained model! Loading weights /content/drive/My Drive/Mask-rcnn1/mask_rcnn_coco.h5 WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:190: The name tf.get_default_session is deprecated. Please use tf.compat.v1.get_default_session instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:197: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:203: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.
2020-01-10 05:15:33.850593: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2300000000 Hz 2020-01-10 05:15:33.852332: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1a5b0bc0 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2020-01-10 05:15:33.852378: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2020-01-10 05:15:33.880022: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2020-01-10 05:15:33.981714: E tensorflow/stream_executor/cuda/cuda_driver.cc:318] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected 2020-01-10 05:15:33.981792: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (b45da54bc5e2): /proc/driver/nvidia/version does not exist WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:207: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:216: The name tf.is_variable_initialized is deprecated. Please use tf.compat.v1.is_variable_initialized instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:223: The name tf.variables_initializer is deprecated. Please use tf.compat.v1.variables_initializer instead.
Traceback (most recent call last): File "balloon.py", line 364, in train(model) File "balloon.py", line 183, in train dataset_train.load_balloon(args.dataset, "train") File "balloon.py", line 112, in load_balloon annotations = json.load(open(os.path.join(dataset_dir, "via_region_data.json"))) FileNotFoundError: [Errno 2] No such file or directory: 'Mask_RCNN master/dataset/train/via_region_data.json'
I tried every possibility by changing the path but it is showing same error as "via_region_data.json" file not found. Please help!
I am trying to train balloon.py file on balloon dataset. Both the train and test datasets contain via_region_data.json file. However the error that I am getting is "via_region_data.json" file not available. Trying to run in google colab with the below command. !python balloon.py train --dataset=Mask_RCNN-master/data/balloon --weights=coco
The error that I am getting is as below:
Using TensorFlow backend. Weights: coco Dataset: Mask_RCNN-master/dataset Logs: /content/drive/My Drive/Mask-rcnn1/logs
Configurations: BACKBONE resnet101 BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 2 BBOX_STD_DEV [0.1 0.1 0.2 0.2] COMPUTE_BACKBONE_SHAPE None DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.9 DETECTION_NMS_THRESHOLD 0.3 FPN_CLASSIF_FC_LAYERS_SIZE 1024 GPU_COUNT 1 GRADIENT_CLIP_NORM 5.0 IMAGES_PER_GPU 2 IMAGE_CHANNEL_COUNT 3 IMAGE_MAX_DIM 1024 IMAGE_META_SIZE 14 IMAGE_MIN_DIM 800 IMAGE_MIN_SCALE 0 IMAGE_RESIZE_MODE square IMAGE_SHAPE [1024 1024 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME balloon NUM_CLASSES 2 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 PRE_NMS_LIMIT 6000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (32, 64, 128, 256, 512) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 100 TOP_DOWN_PYRAMID_SIZE 256 TRAIN_BN False TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:541: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:66: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:4432: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:2139: The name tf.nn.fused_batch_norm is deprecated. Please use tf.compat.v1.nn.fused_batch_norm instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:4267: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:2239: The name tf.image.resize_nearest_neighbor is deprecated. Please use tf.compat.v1.image.resize_nearest_neighbor instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/array_ops.py:1475: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.where in 2.0, which has the same broadcast rule as np.where WARNING:tensorflow:From /content/drive/My Drive/Mask-rcnn1/Mask_RCNN-master/balloon/mrcnn/model.py:553: The name tf.random_shuffle is deprecated. Please use tf.random.shuffle instead.
WARNING:tensorflow:From /content/drive/My Drive/Mask-rcnn1/Mask_RCNN-master/balloon/mrcnn/utils.py:202: The name tf.log is deprecated. Please use tf.math.log instead.
WARNING:tensorflow:From /content/drive/My Drive/Mask-rcnn1/Mask_RCNN-master/balloon/mrcnn/model.py:600: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead Downloading pretrained model to /content/drive/My Drive/Mask-rcnn1/mask_rcnn_coco.h5 ... ... done downloading pretrained model! Loading weights /content/drive/My Drive/Mask-rcnn1/mask_rcnn_coco.h5 WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:190: The name tf.get_default_session is deprecated. Please use tf.compat.v1.get_default_session instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:197: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:203: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.
2020-01-10 05:15:33.850593: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2300000000 Hz 2020-01-10 05:15:33.852332: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1a5b0bc0 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2020-01-10 05:15:33.852378: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2020-01-10 05:15:33.880022: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2020-01-10 05:15:33.981714: E tensorflow/stream_executor/cuda/cuda_driver.cc:318] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected 2020-01-10 05:15:33.981792: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (b45da54bc5e2): /proc/driver/nvidia/version does not exist WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:207: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:216: The name tf.is_variable_initialized is deprecated. Please use tf.compat.v1.is_variable_initialized instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:223: The name tf.variables_initializer is deprecated. Please use tf.compat.v1.variables_initializer instead.
Traceback (most recent call last): File "balloon.py", line 364, in train(model) File "balloon.py", line 183, in train dataset_train.load_balloon(args.dataset, "train") File "balloon.py", line 112, in load_balloon annotations = json.load(open(os.path.join(dataset_dir, "via_region_data.json"))) FileNotFoundError: [Errno 2] No such file or directory: 'Mask_RCNN master/dataset/train/via_region_data.json'
I tried every possibility by changing the path but it is showing same error as "via_region_data.json" file not found. Please help!
iam also stuck in this process...
FileNotFoundError Traceback (most recent call last)
Hi,
I am trying to train for coco dataset. The command I am using is: python3 coco.py train --dataset=/home/mask-rcnn/Mask_RCNN-master/dataset-coco --model=coco --download=false --year=2017
The dataset is already downloaded before. I am getting "FileNotFoundError: [Errno 2] No such file or directory: '/home/mask-rcnn/Mask_RCNN-master/dataset-coco/annotations/instances_valminusminival2017.json' "
Can anyone please help? TIA. Please see the logs below.
Using TensorFlow backend. Command: train Model: coco Dataset: /home/mask-rcnn/Mask_RCNN-master/dataset-coco Year: 2017 Logs: /home/mask-rcnn/Mask_RCNN-master/logs Auto Download: True
Configurations: BACKBONE_SHAPES [[256 256] [128 128] [ 64 64] [ 32 32] [ 16 16]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 2 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.7 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 2 IMAGE_MAX_DIM 1024 IMAGE_MIN_DIM 800 IMAGE_PADDING True IMAGE_SHAPE [1024 1024 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME coco NUM_CLASSES 81 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (32, 64, 128, 256, 512) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001
Loading weights /home/mask-rcnn/Mask_RCNN-master/mask_rcnn_coco.h5 2018-01-11 17:09:33.070611: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA 2018-01-11 17:09:33.262017: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Found device 0 with properties: name: GeForce GTX TITAN X major: 5 minor: 2 memoryClockRate(GHz): 1.076 pciBusID: 0000:01:00.0 totalMemory: 11.91GiB freeMemory: 11.47GiB 2018-01-11 17:09:33.262044: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce GTX TITAN X, pci bus id: 0000:01:00.0, compute capability: 5.2) Will use images in /home/mask-rcnn/Mask_RCNN-master/dataset-coco/train2017 Will use annotations in /home/mask-rcnn/Mask_RCNN-master/dataset-coco/annotations/instances_train2017.json loading annotations into memory... Done (t=12.45s) creating index... index created! Will use images in /home/mask-rcnn/Mask_RCNN-master/dataset-coco/val2017 Will use annotations in /home/mask-rcnn/Mask_RCNN-master/dataset-coco/annotations/instances_valminusminival2014.json loading annotations into memory... Traceback (most recent call last): File "coco.py", line 477, in
dataset_train.load_coco(args.dataset, "valminusminival", year=args.year, auto_download=args.download)
File "coco.py", line 108, in loadcoco
coco = COCO("{}/annotations/instances{}{}.json".format(dataset_dir, subset, year))
File "/usr/local/lib/python3.5/dist-packages/pycocotools/coco.py", line 79, in init
dataset = json.load(open(annotation_file, 'r'))
FileNotFoundError: [Errno 2] No such file or directory: '/home/mask-rcnn/Mask_RCNN-master/dataset-coco/annotations/instances_valminusminival2017.json'