Closed momo1986 closed 5 years ago
I tried to make the validation data and training data with same format:
predictor_sizes = [model.get_layer('conv4_3_norm_mbox_conf').output_shape[1:3],
model.get_layer('fc7_mbox_conf').output_shape[1:3],
model.get_layer('conv6_2_mbox_conf').output_shape[1:3],
model.get_layer('conv7_2_mbox_conf').output_shape[1:3],
model.get_layer('conv8_2_mbox_conf').output_shape[1:3],
model.get_layer('conv9_2_mbox_conf').output_shape[1:3]]
ssd_input_encoder = SSDInputEncoder(img_height=img_height,
img_width=img_width,
n_classes=n_classes,
predictor_sizes=predictor_sizes,
scales=scales,
aspect_ratios_per_layer=aspect_ratios,
two_boxes_for_ar1=two_boxes_for_ar1,
steps=steps,
offsets=offsets,
clip_boxes=clip_boxes,
variances=variances,
matching_type='multi',
pos_iou_threshold=0.5,
neg_iou_limit=0.5,
normalize_coords=normalize_coords)
train_generator = train_dataset.generate(batch_size=batch_size,
shuffle=True,
transformations=[ssd_data_augmentation],
label_encoder=ssd_input_encoder,
returns={'processed_images',
'encoded_labels'},
keep_images_without_gt=False)
val_generator = validation_dataset.generate(batch_size=batch_size,
shuffle=False,
transformations=[ssd_data_augmentation],
label_encoder=ssd_input_encoder,
returns={'processed_images',
'encoded_labels'},
keep_images_without_gt=False)
# Define a learning rate schedule.
train_dataset_size = train_dataset.get_dataset_size()
val_dataset_size = validation_dataset.get_dataset_size()
However, still got such error: Epoch 1/200
Epoch 00001: LearningRateScheduler setting learning rate to 0.001.
19/20 [===========================>..] - ETA: 3s - loss: 2.9068Traceback (most recent call last):
File "train_ssd300.py", line 214, in
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
I am getting the same error, could you tell me how to fix it?
I am getting the same error, could you tell me how to fix it?
Are you sure you're sending the parameters for training in the correct order (xmin, ymin, xmax, ymax)?
yes. I cross-checked the values. When I debugged the code, I can some values in batch_items_to_remove. Could you tell me why I am getting empty batch_y?
If you open a GitHub issue, here is the policy:
Your issue must be about one of the following:
I tried to launch a ssd-300 training network.
After one epoch, a problem happens:
Epoch 1/200
Epoch 00001: LearningRateScheduler setting learning rate to 0.001. 199/200 [============================>.] - ETA: 2s - loss: 3.4431Traceback (most recent call last): File "train_ssd300.py", line 215, in
initial_epoch=initial_epoch)
File "/root/anaconda3/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
return func(*args, *kwargs)
File "/root/anaconda3/lib/python3.6/site-packages/keras/engine/training.py", line 1415, in fit_generator
initial_epoch=initial_epoch)
File "/root/anaconda3/lib/python3.6/site-packages/keras/engine/training_generator.py", line 230, in fit_generator
workers=0)
File "/root/anaconda3/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
return func(args, **kwargs)
File "/root/anaconda3/lib/python3.6/site-packages/keras/engine/training.py", line 1469, in evaluate_generator
verbose=verbose)
File "/root/anaconda3/lib/python3.6/site-packages/keras/engine/training_generator.py", line 327, in evaluate_generator
generator_output = next(output_generator)
File "/root/anaconda3/lib/python3.6/site-packages/keras/utils/data_utils.py", line 793, in get
six.reraise(value.class, value, value.traceback)
File "/root/anaconda3/lib/python3.6/site-packages/six.py", line 693, in reraise
raise value
File "/root/anaconda3/lib/python3.6/site-packages/keras/utils/data_utils.py", line 658, in _data_generator_task
generator_output = next(self._generator)
File "/fast/junyan/HandDetection/ssd_keras/data_generator/object_detection_2d_data_generator.py", line 1291, in generate
"must be homogenous in size along all axes.")
data_generator.object_detection_2d_data_generator.DegenerateBatchError: You produced an empty batch. This might be because the images in the batch vary in their size and/or number of channels. Note that after all transformations (if any were given) have been applied to all images in the batch, all images must be homogenous in size along all axes.
`Epoch 1/200
Epoch 00001: LearningRateScheduler setting learning rate to 0.001. 199/200 [============================>.] - ETA: 2s - loss: 3.4431Traceback (most recent call last): File "train_ssd300.py", line 215, in
initial_epoch=initial_epoch)
File "/root/anaconda3/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
return func(*args, *kwargs)
File "/root/anaconda3/lib/python3.6/site-packages/keras/engine/training.py", line 1415, in fit_generator
initial_epoch=initial_epoch)
File "/root/anaconda3/lib/python3.6/site-packages/keras/engine/training_generator.py", line 230, in fit_generator
workers=0)
File "/root/anaconda3/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
return func(args, **kwargs)
File "/root/anaconda3/lib/python3.6/site-packages/keras/engine/training.py", line 1469, in evaluate_generator
verbose=verbose)
File "/root/anaconda3/lib/python3.6/site-packages/keras/engine/training_generator.py", line 327, in evaluate_generator
generator_output = next(output_generator)
File "/root/anaconda3/lib/python3.6/site-packages/keras/utils/data_utils.py", line 793, in get
six.reraise(value.class, value, value.traceback)
File "/root/anaconda3/lib/python3.6/site-packages/six.py", line 693, in reraise
raise value
File "/root/anaconda3/lib/python3.6/site-packages/keras/utils/data_utils.py", line 658, in _data_generator_task
generator_output = next(self._generator)
File "/fast/junyan/HandDetection/ssd_keras/data_generator/object_detection_2d_data_generator.py", line 1291, in generate
"must be homogenous in size along all axes.")
data_generator.object_detection_2d_data_generator.DegenerateBatchError: You produced an empty batch. This might be because the images in the batch vary in their size and/or number of channels. Note that after all transformations (if any were given) have been applied to all images in the batch, all images must be homogenous in size along all axes.
` My Input data is center-point and width, height,
I do transfer like this: