[Object Detection] Is the AutoAugment option already working?

JonasGlow commented 4 years ago

System information

What is the top-level directory of the model you are using:
Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
TensorFlow installed from (source or binary):
TensorFlow version (use command below):
Bazel version (if compiling from source):
CUDA/cuDNN version:
GPU model and memory:
Exact command to reproduce: Not important.

Is the AutoAugment option in the preprocessor.proto already working? I tried to use it but it always couldn't find the augmentation.

// Apply an Autoaugment policy to the image and bounding boxes.
message AutoAugmentImage {

  // What AutoAugment policy to apply to the Image
  optional string policy_name = 1 [default="v0"];
}

staubda commented 4 years ago

Using the object detection research repo I'm running it by adding the following to my config file (make sure you've compiled your protos recently enough to have it included)

  data_augmentation_options {
    autoaugment_image {
    }
  }

however, I'm hitting the following error:

  File "/home/david_staub/.conda/envs/user/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1334, in_do_call
    return fn(*args)
  File "/home/david_staub/.conda/envs/user/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1319, in_run_fn
    options, feed_dict, fetch_list, target_list, run_metadata)
  File "/home/david_staub/.conda/envs/user/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1407, in_call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: output dimensions must be positive
         [[{{node cond_10/cond/cond/while/cond/translate/transform/ImageProjectiveTransformV2}} = ImageProjectiveTransformV2[dtype=DT_UINT8, interpolation="NEAREST", _device="/device:CPU:0"](cond_10/cond/cond/while/cond/translate/transform/strided_slice, cond_10/cond/cond/while/cond/translate/translations_to_projective_transforms/concat, cond_10/cond/cond/while/cond/translate/transform/strided_slice_2)]]
         [[{{node IteratorGetNext}} = IteratorGetNext[output_shapes=[[24], [24,300,300,3], [24,2], [24,3], [24,100], [24,100,4], [24,100,36], [24,100,36], [24,100], [24,100], [24,100], [24]], output_types=[DT_INT32, DT_FLOAT, DT_INT32, DT_INT32,DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_INT32, DT_BOOL, DT_FLOAT, DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](IteratorV2)]]
         [[{{node GroupCrossDeviceControlEdges_0/ConstantFolding/Loss/Compare_7/IOU/Intersection/split_1-folded-3/_10252}}= _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_3293_...1-folded-3", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]

Also as an FYI, you can figure out the names of augmentation options and their parameters (that you'll need for the config file) by looking in the generated pb2.py file, i.e. models/research/object_detection/protos/preprocessor_pb2.py. Wish this stuff was better documented (or documented at all).

EDIT

Rerunning the training script (didn't change anything) and haven't hit the error so far. Maybe non-deterministic?

EDIT 2

Crashed again after running for a couple thousand steps.

JonasGlow commented 4 years ago

Sorry for the late response. I compiled my protos and i can find the autoaugmentation step in den preprocessor_pb2.py, but i still getting this error.

Message type "object_detection.protos.PreprocessingStep" has no field named "autoaugment_image".

Adblu commented 4 years ago

I can confirm that using most recent object detection api I got that error. Any Fix ?

koi-boy commented 3 years ago