search

tensorflow / tpu

Reference models and tools for Cloud TPUs.
https://cloud.google.com/tpu/
Apache License 2.0
5.2k stars 1.77k forks source link

Learning Data Augmentation Strategies for Object Detection？ #423

Open feixiangdekaka opened 5 years ago

feixiangdekaka commented 5 years ago

Where is the code for implement the 《Learning Data Augmentation Strategies for Object Detection》?

saberkun commented 5 years ago

When using augmentation, it triggers https://github.com/tensorflow/tpu/blob/master/models/official/detection/utils/autoaugment_utils.py#L15

Yaml config is https://github.com/tensorflow/tpu/blob/master/models/official/detection/configs/yaml/retinanet_autoaugment.yaml

gittigxuy commented 5 years ago

So,how to use this autoaugment_util.py file?could you please give me a example?Thanks

heartInsert commented 4 years ago

So,how to use this autoaugment_util.py file?could you please give me a example?Thanks

In detection.dataloader.retinanet_parser.py line 221

` if self._use_autoaugment:

  image, boxes = autoaugment_utils.distort_image_with_autoaugment(
      image, boxes, self._autoaugment_policy_name)`

make self._autoaugment_policy_name='v0'

poodarchu commented 4 years ago

does it work on coco? I found several bugs in autoaugment_utils.py.

BarretZoph commented 4 years ago

Yes it does work on COCO. What bugs did you find in autoaugment_utils.py?

heartInsert commented 4 years ago

I think I didn"t meet bugs here

poodarchu commented 4 years ago

Yes it does work on COCO. What bugs did you find in autoaugment_utils.py?

  1. When it does bbox only flip / shear / translate, if a bbox has box in it, it will cause the smaller bboxes mismatch with correspoinding objects.
  2. if a bbox is out of image range after a transform, the box will still exist in image, and it's shape is totally wrong.
heartInsert commented 4 years ago

Yes it does work on COCO. What bugs did you find in autoaugment_utils.py?

  1. When it does bbox only flip / shear / translate, if a bbox has box in it, it will cause the smaller bboxes mismatch with correspoinding objects.
  2. if a bbox is out of image range after a transform, the box will still exist in image, and it's shape is totally wrong.

I can't reply you the first question,. The second question's answer is that : it's not wrong , this is what augment should do .This situation has been displayed in their paper.

poodarchu commented 4 years ago

Yes it does work on COCO. What bugs did you find in autoaugment_utils.py?

  1. When it does bbox only flip / shear / translate, if a bbox has box in it, it will cause the smaller bboxes mismatch with correspoinding objects.
  2. if a bbox is out of image range after a transform, the box will still exist in image, and it's shape is totally wrong.

I can't reply you the first question,. The second question's answer is that : it's not wrong , this is what augment should do .This situation has been displayed in their paper.

But in my experiments, If I do not fix these 'bugs', mAP on COCO only achieves 10.5%。after I fix these, mAP start to be resonable.

heartInsert commented 4 years ago

Yes it does work on COCO. What bugs did you find in autoaugment_utils.py?

  1. When it does bbox only flip / shear / translate, if a bbox has box in it, it will cause the smaller bboxes mismatch with correspoinding objects.
  2. if a bbox is out of image range after a transform, the box will still exist in image, and it's shape is totally wrong.

I can't reply you the first question,. The second question's answer is that : it's not wrong , this is what augment should do .This situation has been displayed in their paper.

But in my experiments, If I do not fix these 'bugs', mAP on COCO only achieves 10.5%。after I fix these, mAP start to be resonable.

When these two bugs which you said above been applied in training , it means the unstable increased in training data . Did you run more 5 or 10 epoch in training . If MAP in test_data correspoinding increased , it meas it's not a bug , or if MAP doesn't changed ,I think you are right . If you do these experiment above , please inform me.

poodarchu commented 4 years ago

Yes it does work on COCO. What bugs did you find in autoaugment_utils.py?

  1. When it does bbox only flip / shear / translate, if a bbox has box in it, it will cause the smaller bboxes mismatch with correspoinding objects.
  2. if a bbox is out of image range after a transform, the box will still exist in image, and it's shape is totally wrong.

I can't reply you the first question,. The second question's answer is that : it's not wrong , this is what augment should do .This situation has been displayed in their paper.

But in my experiments, If I do not fix these 'bugs', mAP on COCO only achieves 10.5%。after I fix these, mAP start to be resonable.

When these two bugs which you said above been applied in training , it means the unstable increased in training data . Did you run more 5 or 10 epoch in training . If MAP in test_data correspoinding increased , it meas it's not a bug , or if MAP doesn't changed ,I think you are right . If you do these experiment above , please inform me.

Have you tried on COCO?

heartInsert commented 4 years ago

Yes it does work on COCO. What bugs did you find in autoaugment_utils.py?

  1. When it does bbox only flip / shear / translate, if a bbox has box in it, it will cause the smaller bboxes mismatch with correspoinding objects.
  2. if a bbox is out of image range after a transform, the box will still exist in image, and it's shape is totally wrong.

I can't reply you the first question,. The second question's answer is that : it's not wrong , this is what augment should do .This situation has been displayed in their paper.

But in my experiments, If I do not fix these 'bugs', mAP on COCO only achieves 10.5%。after I fix these, mAP start to be resonable.

When these two bugs which you said above been applied in training , it means the unstable increased in training data . Did you run more 5 or 10 epoch in training . If MAP in test_data correspoinding increased , it meas it's not a bug , or if MAP doesn't changed ,I think you are right . If you do these experiment above , please inform me.

Have you tried on COCO?

COCO is a little large for my 1050ti （：

poodarchu commented 4 years ago

I've convert the tf version to numpy, so everyone can run using any dl framework.

https://github.com/poodarchu/learn_aug_for_object_detection.numpy

heartInsert commented 4 years ago

Great work ,thanks for sharing

sefira commented 4 years ago

https://github.com/tensorflow/tpu/blob/master/models/official/detection/utils/autoaugment_utils.py#L686

@BarretZoph @aman2930 @saberkun I think I've found another bug for Learning Data Augmentation Strategies for Object Detection seen above link, in which an image is augmented as many time as the number of bboxes in it. Assuming we are augmenting the bboxes and images by random flipping, in this case, each bbox is flipped by a probability P, but the image will be flip many times and the probability is not equal to P.

BarretZoph commented 4 years ago

Yes it does work on COCO. What bugs did you find in autoaugment_utils.py?

  1. When it does bbox only flip / shear / translate, if a bbox has box in it, it will cause the smaller bboxes mismatch with correspoinding objects.
  2. if a bbox is out of image range after a transform, the box will still exist in image, and it's shape is totally wrong.

Yes these two things do occur, but the performance is still good on COCO. Some of these could actually potentially benefit the training by acting as a regularizer.

I have since fixed 2 and noticed that on COCO the performance on ResNet-50 improves by about 0 to 0.2 mAP depending on the augmentation policy. I am playing around with both fixes to see if the performance could be better improved still, but the performance in the paper is obtained with both of these features in it.

BarretZoph commented 4 years ago

https://github.com/tensorflow/tpu/blob/master/models/official/detection/utils/autoaugment_utils.py#L686

@BarretZoph @aman2930 @saberkun I think I've found another bug for Learning Data Augmentation Strategies for Object Detection seen above link, in which an image is augmented as many time as the number of bboxes in it. Assuming we are augmenting the bboxes and images by random flipping, in this case, each bbox is flipped by a probability P, but the image will be flip many times and the probability is not equal to P.

I am not sure I understand this. There operations that are applied I.I.D to each bbox with the image.

Jacobew commented 4 years ago

Yes it does work on COCO. What bugs did you find in autoaugment_utils.py?

  1. When it does bbox only flip / shear / translate, if a bbox has box in it, it will cause the smaller bboxes mismatch with correspoinding objects.
  2. if a bbox is out of image range after a transform, the box will still exist in image, and it's shape is totally wrong.

I can't reply you the first question,. The second question's answer is that : it's not wrong , this is what augment should do .This situation has been displayed in their paper.

But in my experiments, If I do not fix these 'bugs', mAP on COCO only achieves 10.5%。after I fix these, mAP start to be resonable.

@poodarchu What AP did you get when these bugs got fixed? BTW, could you share the fixed version? Thanks.

BarretZoph commented 4 years ago

Just to clarify, with the two issues listed above, you should be able to reproduce the results in the original paper. When fixing them I found little to no mAP performance improvement on COCO.

Qianshaowei commented 3 years ago

When using augmentation, it triggers https://github.com/tensorflow/tpu/blob/master/models/official/detection/utils/autoaugment_utils.py#L15

Yaml config is https://github.com/tensorflow/tpu/blob/master/models/official/detection/configs/yaml/retinanet_autoaugment.yaml

Excuse me, where is the code of the RNN controller?

noreenanwar commented 1 year ago

This method works for small objects?