Open JaydenKing32 opened 3 years ago
JaydenKing32
commented
3 years ago Ah, it seems as though changing h.num_classes was the problem. If I set h.label_map = {1: 'person', 2: 'dog'} and leave h.num_classes alone, then it seems to work fine.
I suppose I fixed my issue, though it doesn't feel like my method is the intended solution.
matankley
commented
3 years ago @JaydenKing32 Can you please describe the method that worked for you ? Keeping h.num_classes as coco default and only change h.label_map to your custom dataset ? Thanks
JaydenKing32
commented
3 years ago @matankley
Yes, I didn't change the number of classes, it was left as h.num_classes = 90.
I only changed the label map to h.label_map = {1: 'person', 2: 'dog'}, and the per-class AP scores for "person" and "dog" were produced alongside the other results.
alexlorenzo
commented
3 years ago Hello @JaydenKing32 , did you find a cleaner solution for h.num_classes = 90?
We have the same issue.
Thanks
Hello, I've been trying to get per-class AP results for evaluation, but I can't figure out how to make it work when evaluating a new dataset or setting specific classes with the COCO 2017 dataset.
I found out via e03aaee that I could change
h.label_map = Nonetoh.label_map = 'coco'in hparams_config.py in order to produce per-class AP results for all 90 COCO classes when evaluating the COCO 2017 dataset. However, when I tried to specify classes using this method I ended up with an "Incompatible shapes" error, (full log is included below). This attempt was made by settingh.num_classes = 3andh.label_map = {1: 'person', 2: 'dog'}with the COCO 2017 dataset, but I would like to be able to specify classes for other datasets as well.Is there some other method of specifying classes that I am not aware of, or is this currently not possible? I've also seen
label_mapbeing set via thehparamsoption, but only for training. I've attempted to use hparams when evaluating with a pre-trained model, but this also resulted in an error.Additionally, I believe this may be related to f701a0b and #762 which also seem to involve
label_map.Log
``` > python main.py --mode=eval --model_name=efficientdet-d0 --model_dir=models/efficientdet-d0/ --validation_file_pattern=tfrecord/val* --val_json_file=data/annotations/instances_val2017.json 2020-10-27 04:14:09.954175: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'cudart64_101.dll'; dlerror: cudart64_101.dll not found 2020-10-27 04:14:09.954425: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine. I1027 04:14:15.212893 297028 main.py:228] {'name': 'efficientdet-d0', 'act_type': 'swish', 'image_size': (512, 512), 'target_size': None, 'input_rand_hflip': True, 'jitter_min': 0.1, 'jitter_max': 2.0, 'autoaugment_policy': None, 'use_augmix': False, 'grid_mask': False, 'augmix_params': [3, -1, 1], 'sample_image': None, 'num_classes': 3, 'seg_num_classes': 3, 'heads': ['object_detection'], 'skip_crowd_during_training': True, 'label_map': {1: 'person', 2: 'dog'}, 'max_instances_per_image': 100, 'regenerate_source_id': False, 'min_level': 3, 'max_level': 7, 'num_scales': 3, 'aspect_ratios': [1.0, 2.0, 0.5], 'anchor_scale': 4.0, 'is_training_bn': True, 'momentum': 0.9, 'optimizer': 'sgd', 'learning_rate': 0.08, 'lr_warmup_init': 0.008, 'lr_warmup_epoch': 1.0, 'first_lr_drop_epoch': 200.0, 'second_lr_drop_epoch': 250.0, 'poly_lr_power': 0.9, 'clip_gradients_norm': 10.0, 'num_epochs': 300, 'data_format': 'channels_last', 'label_smoothing': 0.0, 'alpha': 0.25, 'gamma': 1.5, 'delta': 0.1, 'box_loss_weight': 50.0, 'iou_loss_type': None, 'iou_loss_weight': 1.0, 'weight_decay': 4e-05, 'strategy': None, 'mixed_precision': False, 'box_class_repeats': 3, 'fpn_cell_repeats': 3, 'fpn_num_filters': 64, 'separable_conv': True, 'apply_bn_for_resampling': True, 'conv_after_downsample': False, 'conv_bn_act_pattern': False, 'drop_remainder': True, 'nms_configs': {'method': 'gaussian', 'iou_thresh': None, 'score_thresh': None, 'sigma': None, 'max_nms_inputs': 0, 'max_output_size': 100}, 'fpn_name': None, 'fpn_weight_method': None, 'fpn_config': None, 'survival_prob': None, 'img_summary_steps': None, 'lr_decay_method': 'cosine', 'moving_average_decay': 0.9998, 'ckpt_var_scope': None, 'skip_mismatch': True, 'backbone_name': 'efficientnet-b0', 'backbone_config': None, 'var_freeze_expr': None, 'use_keras_model': True, 'dataset_type': None, 'positives_momentum': None, 'device': {'grad_ckpting': False, 'grad_ckpting_list': ['Add_', 'AddN'], 'nvgpu_logging': False}, 'model_name': 'efficientdet-d0', 'iterations_per_loop': 100, 'model_dir': 'models/efficientdet-d0/', 'num_shards': 8, 'num_examples_per_epoch': 120000, 'backbone_ckpt': '', 'ckpt': None, 'val_json_file': 'data/annotations/instances_val2017.json', 'testdev_dir': None, 'profile': False, 'mode': 'eval'} INFO:tensorflow:Using config: {'_model_dir': 'models/efficientdet-d0/', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': 100, '_save_checkpoints_secs': None, '_session_config': allow_soft_placement: true, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1} INFO:tensorflow:Waiting for new checkpoint at models/efficientdet-d0/ I1027 04:14:15.285697 297028 checkpoint_utils.py:125] Waiting for new checkpoint at models/efficientdet-d0/ INFO:tensorflow:Found new checkpoint at models/efficientdet-d0/model I1027 04:14:15.291682 297028 checkpoint_utils.py:134] Found new checkpoint at models/efficientdet-d0/model I1027 04:14:15.293684 297028 main.py:308] Starting to evaluate. 2020-10-27 04:14:15.446721: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library nvcuda.dll 2020-10-27 04:14:15.477384: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties: pciBusID: 0000:01:00.0 name: GeForce GTX 970 computeCapability: 5.2 coreClock: 1.342GHz coreCount: 13 deviceMemorySize: 4.00GiB deviceMemoryBandwidth: 211.48GiB/s 2020-10-27 04:14:15.480269: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'cudart64_101.dll'; dlerror: cudart64_101.dll not found 2020-10-27 04:14:15.483283: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'cublas64_10.dll'; dlerror: cublas64_10.dll not found 2020-10-27 04:14:15.491425: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'cufft64_10.dll'; dlerror: cufft64_10.dll not found 2020-10-27 04:14:15.494261: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'curand64_10.dll'; dlerror: curand64_10.dll not found 2020-10-27 04:14:15.496957: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'cusolver64_10.dll'; dlerror: cusolver64_10.dll not found 2020-10-27 04:14:15.499897: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'cusparse64_10.dll'; dlerror: cusparse64_10.dll not found 2020-10-27 04:14:15.507223: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'cudnn64_7.dll'; dlerror: cudnn64_7.dll not found 2020-10-27 04:14:15.507373: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1753] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform. Skipping registering GPU devices... INFO:tensorflow:Calling model_fn. I1027 04:14:15.834230 297028 estimator.py:1162] Calling model_fn. I1027 04:14:15.843206 297028 efficientnet_builder.py:215] global_params= GlobalParams(batch_norm_momentum=0.99, batch_norm_epsilon=0.001, dropout_rate=0.2, data_format='channels_last', num_classes=1000, width_coefficient=1.0, depth_coefficient=1.0, depth_divisor=8, min_depth=None, survival_prob=0.0, relu_fn=functools.partial(