Build R-CNN with ResNet

[JihongJu]

• Build an RCNN model with keras_resnet.ResNet50.
• Implement heads described in Mask R-CNN.

My current implementation is:

        y = keras_resnet.ResNet50(inputs)
        features = y.layers[-2].output

        rpn_classification = keras.layers.Conv2D(
            9 * 1, (1, 1), activation="sigmoid")(features)
        rpn_regression = keras.layers.Conv2D(9 * 4, (1, 1))(features)

        rpn_prediction = keras.layers.concatenate(
            [rpn_classification, rpn_regression])

        # proposals of shape (1, None, 4)
        proposals = keras_rcnn.layers.object_detection.ObjectProposal(
            rois)([rpn_regression, rpn_classification])

        # slices of shape (1, None, 7, 7, 3)
        slices = keras_rcnn.layers.ROI((7, 7), rois)([inputs, proposals])

        # Implement the R-CNN heads shown in the figure below
        [score, boxes] = keras_rcnn.heads.ResHead(classes)(slices)

And the R-CNN heads from Mask R-CNN looks like

That becomes:

        y = keras.layers.TimeDistributed(
            keras.layers.Conv2D(1024, (1, 1)))(x)

        # conv5 block as in Deep Residual Networks with first conv operates
        # on a 7x7 RoI with stride 1 (instead of 14x14 / stride 2)
        for i in range(3):
            y = _bottleneck(512, (1, 1))(y)

        y = keras.layers.TimeDistributed(
            keras.layers.BatchNormalization(axis=channel_axis))(y)
        y = keras.layers.TimeDistributed(
            keras.layers.Activation("relu"))(y)

        # class and box branches
        y = keras.layers.TimeDistributed(
            keras.layers.AveragePooling2D((7, 7)))(y)

        score = keras.layers.TimeDistributed(
            keras.layers.Dense(classes, activation="softmax"))(y)

        boxes = keras.layers.TimeDistributed(
            keras.layers.Dense(4 * classes))(y)

in keras_rcnn.heads.ResHead.

What do you think? @0x00b1

P.S. The API design should be discussed in broadinstitute/keras-rcnn#28

[codecov-io]

Codecov Report

Merging #27 into master will increase coverage by 4.81%. The diff coverage is 97.14%.

@@            Coverage Diff             @@
##           master      #27      +/-   ##
==========================================
+ Coverage   48.44%   53.25%   +4.81%     
==========================================
  Files          15       17       +2     
  Lines         545      569      +24     
==========================================
+ Hits          264      303      +39     
+ Misses        281      266      -15

Impacted Files	Coverage Δ
keras_rcnn/heads/__init__.py	100% <100%> (ø)
keras_rcnn/models.py	81.48% <100%> (+81.48%)	:arrow_up:
keras_rcnn/heads/resnet.py	94.11% <94.11%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update f3e6f4c...9aadff0. Read the comment docs.

[0x00b1]

@JihongJu Whoa! You’ve been busy! I’m excited to comment. Would you mind giving me a day or two? It’ll take some time to consider and respond.

[0x00b1]

cc: @mcquin

[0x00b1]

@JihongJu I merged the backend and pooling changes.

[JihongJu]

Great. I will continue the work on weekends.

[JihongJu]

@0x00b1 If you are okay with this design, I will continue with the Mask branch as in TODO with a new PR.

[0x00b1]

@JihongJu Awesome! Merged! (I might do a little cleanup today or tomorrow.)