search

google / automl

Google Brain AutoML
Apache License 2.0
6.24k stars 1.45k forks source link

Training bounding boxes and segmentation masks simultaneously #653

Open robert-hardwick opened 4 years ago

robert-hardwick commented 4 years ago

I am trying to train an efficientdet-d0 model to output bounding boxes and segmentation with the newly merged code from #599

I am attempting to train COCO from scratch

python3 main.py --mode train_and_eval --training_file_pattern /data/tfrecord/train/*.tfrecord --validation_file_pattern /data/tfrecord/val/*.tfrecord --model_name efficientdet-d0 --model_dir /data/efficientdet_run1/ --train_batch_size 32 --eval_batch_size 32

however I run into the following error

tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found. (0) Invalid argument: Cannot batch tensors with different shapes in component 18. First element had shape [0,640,428] and element 1 had shape [0,427,640]. [[node IteratorGetNext (defined at /usr/local/lib/python3.6/dist-packages/tensorflow_estimator/python/estimator/util.py:61) ]] (1) Invalid argument: Cannot batch tensors with different shapes in component 18. First element had shape [0,640,428] and element 1 had shape [0,427,640]. [[node IteratorGetNext (defined at /usr/local/lib/python3.6/dist-packages/tensorflow_estimator/python/estimator/util.py:61) ]] [[IteratorGetNext/_11517]]

This error only appears when 'segmentation' is included in heads, and doesn't appear when batch size is set to 1. So my assumption is that the mask images are of varying sizes and should be reshaped or padded. Have i missed something? Or is this workflow not supported yet?

robert-hardwick commented 4 years ago

UPDATE: I've just merged in #651 and now it is training.

robert-hardwick commented 4 years ago

Ignore previous comment, have misunderstood how this can be used. Seems like segmentation head is only available in kerass model. Plead advise how i can preload weights into keras model for segmentation.

zishanahmed08 commented 4 years ago

@robert-hardwick where you abel to solve your issue?

robert-hardwick commented 4 years ago

hI @zishanahmed08. Was able to get the model to train simultaneously, however could only achieve this by loading weights into backbone only.

model.backbone.load_weights("/path/to/efficientnet-cpkt/")

Now i have the problem of exporting / saving the model.

zishanahmed08 commented 4 years ago

@robert-hardwick Is it a semantic segmentation model or an instance segmentation?

dimimal commented 3 years ago

@robert-hardwick Is it a semantic segmentation model or an instance segmentation?

I have the same question as well. I would appreciate if someone can give an answer to that.