keypoints detect

[HappyKerry]

I want to detect object like license plate by using parallelogram， how to detect the four keypoints？

[zungam]

Dont use mask rcnn. Train on standard CNN like resNet101/VGG16/AlexNet/GoogLeNet and output the four corners.

Or better, do it in two steps. Use Yolo, or Faster RCNN or SDD to detect license plate, then make a super small CNN to detect the corners.

[HappyKerry]

Thanks for your advice！ Is it possible to use the Faster RCNN to detect both the license plate and the four corners?

[zungam]

Yes, faster rcnn creates a feature map before region of interests are fed into a top header for classification and bounding box estimation. It means that you can make your own header which works in a same fashion and takes in region of interests and output the four corners. The good part of having the feature map pre calculated is that you can have a really small header which don't do much work as most is done in the feature map (given that you back propegate your corners error into the feature map extractor). I would recommend just using regular faster rcnn, copy the bounding box header, and start from there. The bounding box header is four values as output, but you need eight values to have both x, y positions. Remember to feed this output labels with real corner positions during training.

[HappyKerry]

@zungam I am not familiar with the faster rcnn code. Can you explain it in more detail? Are there some similar examples?

[zungam]

You need basic understanding of Keras / Tensorflow / Pytorch / caffe or something similar if are du modify an architecture already running on one of these. Also, I recomend reading how faster rcnn works by googling around!

[rbavery]

@HappyKerry this continues to be a helpful article for me. Good summary and citations https://blog.athelas.com/a-brief-history-of-cnns-in-image-segmentation-from-r-cnn-to-mask-r-cnn-34ea83205de4

[HappyKerry]

@zungam @rbavery Thanks

[SwetaKaman]

@zungam hello sir, can you please help me with finding the feature vector of detected objects in mask rcnn or what changes do I have to make in the model.py file so that I can extract the feature vector directly?? Your help will be mu h appreciated..