Can Object detection made possible using laqr?

[Movindsouza]

Feature motivation

Due to the smaller size and lower compute time can this be implemented in the application of object detection?

Feature description

Feature implementation

[jneeven]

Hi, thank you for your interest in Larq! Object detection with BNNs is definitely possible, and there are a few papers on the topic. For example, the BinaryDenseNet paper has a short section on object detection (table 8 on page 8), and they obtain 68.2 mAP on the VOC2007 dataset by using their BinaryDenseNet45 network (which is available in Larq Zoo!) as the backbone for an SSD object detector.

There is also a short section on object detection in Structured Binary Neural Networks for ImageRecognition and BiDet: An Efficient Binarized Object Detector is entirely devoted to object detection with BNNs, though I personally find these papers less impressive and harder to read.

In general, there is no reason you wouldn't be able to use an object detection framework (such as YOLO, SSD or Faster R-CNN) and use a BNN as the feature extraction network. It is definitely possible to create these frameworks in Larq, but object detection frameworks tend to be a bit more complex than simple image recognition applications. Since Larq layers are interchangeable with Keras layers, I therefore suggest having a look at the many Keras object detection repositories on GitHub, e.g. this RetinaNet repo or this YOLOv3 repo, and simply replacing the feature extraction networks they use with binary ones from our larq-zoo library (I have no idea how good these repositories are, these are just the ones I could find with a quick search).

Some final tips I can give you is to use the most accurate BNN that suits your requirements, as a better feature extraction network usually leads to better object detection accuracy (i.e. AP or mAP), and to make sure that the data augmentation is similar to the data augmentation that was used to train the model (e.g. you don't want to feed your pre-trained model inputs in range [0, 1] if it was trained with inputs in range [-1, 1]). It is also likely that if you change a full-precision model for a BNN, you will have to experiment with changing the learning rate a bit.

I hope this answers your question, feel free to reach out to us on our Spectrum chat if you have any more questions!

[Movindsouza]

@jneeven Thank you for your kind response. let's discuss this over spectrum chat if anyone has already implemented it.

[nathankopp]

Quick follow-up here.... is anyone aware of any existing implementations of YOLO, SSD, or a similar one-shot object detector built on Larq? If not, I plan to give it a shot myself... but I'm hoping someone else has done it already so I can save some time.

[lgeiger]

Quick follow-up here.... is anyone aware of any existing implementations of YOLO, SSD, or a similar one-shot object detector built on Larq? If not, I plan to give it a shot myself... but I'm hoping someone else has done it already so I can save some time.

That's great! Since Larq is fully compatible with Keras, I'd recommend looking at https://keras.io/examples/vision/retinanet/ or similar and then replace the backbone model with a binary model built using Larq layers.

[rameshKrSah]

Quick follow-up here.... is anyone aware of any existing implementations of YOLO, SSD, or a similar one-shot object detector built on Larq? If not, I plan to give it a shot myself... but I'm hoping someone else has done it already so I can save some time.

I am also exploring object detection with Larq. Were you able to achieve this?

[jneeven]

I am also exploring object detection with Larq. Were you able to achieve this?

@tehtea has provided a YOLO implementation: https://github.com/tehtea/QuickYOLO (see also https://github.com/larq/compute-engine/pull/614)

[nathankopp]

@rameshKrSah , As @jneeven points out, QuickYOLO is a good implementation. I have used it successfully.

[rameshKrSah]

@jneeven thanks. I will check it out 😄