Closed ZER-0-NE closed 4 years ago
@k-w-w Any comments?
@ZER-0-NE Thanks for the ping. Assigning to the codeowners of the object detection model.
Any comments? @derekjchow
please pass the fulll code,
have a look here tion\trainer.py", line 181, in _create_losses losses_dict = detection_model.loss(prediction_dict, true_image_shapes)
--I think you have to use prediction_dict['XXX'],I had the same problem and solve it using the dictionary index. --a second advice, print (prediction_dict) and print( true_image_shapes) and check if both tensors are of the same type
--I think you have to use prediction_dict['XXX'],I had the same problem and solve it using the dictionary index.
I do not get this. My trainer.py script runs fine. @MaisaDaoud
My personal workaround on this issue, just replace the None with -1 and force the np dtype to be np.int32.
Add following code to the line before this line: https://github.com/tensorflow/tensorflow/blob/42ca99b5aae03a8122ba0db94abfe1f3f5c257dc/tensorflow/python/framework/tensor_util.py#L377
if np_dt==None and len(np.array(values).shape)>0 and None in values:
np_dt = np.int32
values = [-1 if type(v)==type(None) else v for v in values]
['XXX']
What do we put instead of ['XXX] ?
['XXX']
What do we put instead of ['XXX] ?
Did this solution work? @Aaronreb
Hi There, We are checking to see if you still need help on this, as this seems to be considerably old issue. Please update this issue with the latest information, code snippet to reproduce your issue and error you are seeing. If we don't hear from you in the next 7 days, this issue will be closed automatically. If you don't need help on this issue any more, please consider closing this.
python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)": b'unknown' 1.5.0
System information Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 7 x64 Python version: 3.5.2 Exact command to reproduce: python train.py --logtostderr --train_dir=training/ --pipeline_config_path=training/faster_rcnn_nas_coco.config
Describe the problem
I am trying to train my own dataset using faster_rcnn_nas_coco model. I get this unknown error. Other issues which I think were similar could not help: #11974
Source code / logs
Here is the traceback of the error I get:
C:\Users\User\Desktop\Abhishek_Singh\models-master\research>python train.py --lo gtostderr --train_dir=training/ --pipeline_config_path=training/faster_rcnnnas coco.config INFO:tensorflow:Scale of 0 disables regularizer. INFO:tensorflow:Scale of 0 disables regularizer. WARNING:tensorflow:From C:\Users\User\Desktop\Abhishek_Singh\models-master\resea rch\object_detection\trainer.py:228: create_global_step (from tensorflow.contrib .framework.python.ops.variables) is deprecated and will be removed in a future v ersion. Instructions for updating: Please switch to tf.train.create_global_step INFO:tensorflow:Scale of 0 disables regularizer. INFO:tensorflow:Scale of 0 disables regularizer. 2018-04-04 18:41:53.436438: I C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\3 5\tensorflow\core\platform\cpu_feature_guard.cc:137] Your CPU supports instructi ons that this TensorFlow binary was not compiled to use: AVX AVX2 2018-04-04 18:41:53.904439: I C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\3 5\tensorflow\core\common_runtime\gpu\gpu_device.cc:1105] Found device 0 with pro perties: name: Quadro K620 major: 5 minor: 0 memoryClockRate(GHz): 1.124 pciBusID: 0000:02:00.0 totalMemory: 2.00GiB freeMemory: 1.80GiB 2018-04-04 18:41:53.920039: I C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\3 5\tensorflow\core\common_runtime\gpu\gpu_device.cc:1195] Creating TensorFlow dev ice (/device:GPU:0) -> (device: 0, name: Quadro K620, pci bus id: 0000:02:00.0, compute capability: 5.0) INFO:tensorflow:A GPU is available on the machine, consider using NCHW data form at for increased speed on GPU. INFO:tensorflow:depth of additional conv before box predictor: 0 WARNING:tensorflow:From C:\Users\User\Desktop\Abhishek_Singh\models-master\resea rch\object_detection\core\box_predictor.py:396: calling reduce_mean (from tensor flow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version. Instructions for updating: keep_dims is deprecated, use keepdims instead WARNING:tensorflow:From C:\Users\User\Desktop\Abhishek_Singh\models-master\resea rch\object_detection\core\losses.py:316: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future ver sion. Instructions for updating:
Future major versions of TensorFlow will allow gradients to flow into the labels input on backprop by default.
See tf.nn.softmax_cross_entropy_with_logits_v2.
Traceback (most recent call last): File "C:\Program Files (x86)\Python\Python35\lib\site-packages\tensorflow\pyth on\framework\tensor_util.py", line 498, in make_tensor_proto str_values = [compat.as_bytes(x) for x in proto_values] File "C:\Program Files (x86)\Python\Python35\lib\site-packages\tensorflow\pyth on\framework\tensor_util.py", line 498, in
str_values = [compat.as_bytes(x) for x in proto_values]
File "C:\Program Files (x86)\Python\Python35\lib\site-packages\tensorflow\pyth
on\util\compat.py", line 65, in as_bytes
(bytes_or_text,))
TypeError: Expected binary or unicode string, got -1
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "train.py", line 167, in
tf.app.run()
File "C:\Program Files (x86)\Python\Python35\lib\site-packages\tensorflow\pyth
on\platform\app.py", line 124, in run
_sys.exit(main(argv))
File "train.py", line 163, in main
worker_job_name, is_chief, FLAGS.train_dir)
File "C:\Users\User\Desktop\Abhishek_Singh\models-master\research\object_detec
tion\trainer.py", line 246, in train
clones = model_deploy.create_clones(deploy_config, model_fn, [input_queue])
File "C:\Users\User\Desktop\Abhishek_Singh\models-master\research\slim\deploym
ent\model_deploy.py", line 193, in create_clones
outputs = model_fn(*args, kwargs)
File "C:\Users\User\Desktop\Abhishek_Singh\models-master\research\object_detec
tion\trainer.py", line 181, in _create_losses
losses_dict = detection_model.loss(prediction_dict, true_image_shapes)
File "C:\Users\User\Desktop\Abhishek_Singh\models-master\research\object_detec
tion\meta_architectures\faster_rcnn_meta_arch.py", line 1580, in loss
groundtruth_masks_list,
File "C:\Users\User\Desktop\Abhishek_Singh\models-master\research\object_detec
tion\meta_architectures\faster_rcnn_meta_arch.py", line 1773, in _loss_box_class
ifier
weights=batch_cls_weights),
File "C:\Users\User\Desktop\Abhishek_Singh\models-master\research\object_detec
tion\core\losses.py", line 73, in call
return self._compute_loss(prediction_tensor, target_tensor, params)
File "C:\Users\User\Desktop\Abhishek_Singh\models-master\research\object_detec
tion\core\losses.py", line 315, in _compute_loss
labels=tf.reshape(target_tensor, [-1, num_classes]),
File "C:\Program Files (x86)\Python\Python35\lib\site-packages\tensorflow\pyth
on\ops\gen_array_ops.py", line 5184, in reshape
"Reshape", tensor=tensor, shape=shape, name=name)
File "C:\Program Files (x86)\Python\Python35\lib\site-packages\tensorflow\pyth
on\framework\op_def_library.py", line 513, in _apply_op_helper
raise err
File "C:\Program Files (x86)\Python\Python35\lib\site-packages\tensorflow\pyth
on\framework\op_def_library.py", line 510, in _apply_op_helper
preferred_dtype=default_dtype)
File "C:\Program Files (x86)\Python\Python35\lib\site-packages\tensorflow\pyth
on\framework\ops.py", line 1022, in internal_convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "C:\Program Files (x86)\Python\Python35\lib\site-packages\tensorflow\pyth
on\framework\constant_op.py", line 233, in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
File "C:\Program Files (x86)\Python\Python35\lib\site-packages\tensorflow\pyth
on\framework\constant_op.py", line 212, in constant
value, dtype=dtype, shape=shape, verify_shape=verify_shape))
File "C:\Program Files (x86)\Python\Python35\lib\site-packages\tensorflow\pyth
on\framework\tensor_util.py", line 502, in make_tensor_proto
"supported type." % (type(values), values))
TypeError: Failed to convert object of type <class 'list'> to Tensor. Contents:
[-1, None]. Consider casting elements to a supported type.
Here is my config file: model { faster_rcnn { num_classes: 12 image_resizer { fixed_shape_resizer { height: 300 width: 300 } } feature_extractor { type: 'faster_rcnn_nas' } first_stage_anchor_generator { grid_anchor_generator { scales: [0.25, 0.5, 1.0, 2.0] aspect_ratios: [0.5, 1.0, 2.0] height_stride: 16 width_stride: 16 } } first_stage_box_predictor_conv_hyperparams { op: CONV regularizer { l2_regularizer { weight: 0.0 } } initializer { truncated_normal_initializer { stddev: 0.01 } } } first_stage_nms_score_threshold: 0.0 first_stage_nms_iou_threshold: 0.7 first_stage_max_proposals: 300 first_stage_localization_loss_weight: 2.0 first_stage_objectness_loss_weight: 1.0 initial_crop_size: 17 maxpool_kernel_size: 1 maxpool_stride: 1 second_stage_box_predictor { mask_rcnn_box_predictor { use_dropout: false dropout_keep_probability: 1.0 fc_hyperparams { op: FC regularizer { l2_regularizer { weight: 0.0 } } initializer { variance_scaling_initializer { factor: 1.0 uniform: true mode: FAN_AVG } } } } } second_stage_post_processing { batch_non_max_suppression { score_threshold: 0.0 iou_threshold: 0.6 max_detections_per_class: 100 max_total_detections: 100 } score_converter: SOFTMAX } second_stage_localization_loss_weight: 2.0 second_stage_classification_loss_weight: 1.0 second_stage_batch_size: 1 } }
train_config: { batch_size: 1 optimizer { momentum_optimizer: { learning_rate: { manual_step_learning_rate { initial_learning_rate: 0.0003 schedule { step: 0 learning_rate: .0003 } schedule { step: 900000 learning_rate: .00003 } schedule { step: 1200000 learning_rate: .000003 } } } momentum_optimizer_value: 0.9 } use_moving_average: false } gradient_clipping_by_norm: 10.0 fine_tune_checkpoint: faster_rcnn_nas_coco_2018_01_28/model.ckpt" from_detection_checkpoint: true num_steps: 200000 data_augmentation_options { random_horizontal_flip { } } }
train_input_reader: { tf_record_input_reader { input_path: "data/train.record" } label_map_path: "data/CFPS.pbtxt" }
eval_config: { metrics_set: "pascal_voc_metrics" num_examples: 8000 max_evals: 10}
eval_input_reader: { tf_record_input_reader { input_path: "data/test.record" } label_map_path: "training/CFPS.pbtxt" shuffle: false num_readers: 1 num_epochs: 1 }