Things aren't precise.

[nomaanhere]

I can't find what is ultimately being run. I am working on a similar project, but wanted to integrate TF's SSD model on my Raspberry Pi4. Can you be more clear with your Readme.md section.

[leigh-johnson]

Hi @nomaanhere - can you tell me more about what's unclear to you?

The models used in this project are downloadable as tarballs here: https://github.com/leigh-johnson/rpi-deep-pantilt/releases/tag/v1.0.1

If you clone the repo, these tarballs are also checked into the models/ directory.

ssd_mobilenet_v3_small_coco

This is a 32-bit floating point model, based on the ssd_mobilenet_v3_small_coco model in TensorFlow's object detection model zoo. Input values are normalized to 32-bit floats between (-1, 1).

You can inspect models in the zoo here: https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md

ssdlite_mobilenet_edgetpu_coco_quant

This is a quantized model, which takes 8-bit uint between (0 - 255) as input. This model will make faster inferences than the 32-bit model, and it can be accelerated using an Edge TPU. The trade-off is that the accuracy scores tend to me much lower.

Post-processing

I've added one additional layer to the models from the zoo, named TFLite_Detection_PostProcess. This operation is a non-maximum suppression implementation, which takes raw box encodings and class predictions as input and outputs the "best of" bounding box prediction for each class. I recommend this article for a quick intro to NMS algorithms.

TFLite_Detection_PostProcess op definition (inspect inputs / outputs) https://github.com/tensorflow/tensorflow/blob/13dd95a2aec1647c1517ee26b69b1178c502f9ac/tensorflow/compiler/mlir/lite/python/graphdef_to_tfl_flatbuffer.cc#L45

Actual op implementation https://github.com/tensorflow/tensorflow/blob/a1aa33532810dc16c8338afe6649cbd4ec1787f9/tensorflow/lite/kernels/detection_postprocess.cc

Hope this helps!

[nomaanhere]

I followed the entire tutorial of PyImagesearch by Adrian. Over there he used Haarcascade model for detection of faces. I am trying to integrate tensorflow models like ssd(The one you're using). But by following his tutorial I see he uses a command , "$ python pan_tilt_tracking.py --cascade haarcascade_frontalface_default.xml" I trained my own model on tensrflow, but I don't have any .xml file in it. I am quite confused. By seeing your repo I see we are having same line of interest. Can you please help me at this point?

[leigh-johnson]

@nomaanhere Adrian's tutorial does not use TensorFlow, he uses OpenCV. haarcascade_frontalface_default.xml is a reference implementation of a Haar Cascade, which is a technique for detecting edge pixels in image data.

https://github.com/opencv/opencv/blob/master/data/haarcascades/haarcascade_frontalface_default.xml

I'm not using OpenCV or the Haar Cascade technique, so I can't help you out there. Sorry!

Have you tried reading through the instructions in this Medium post? https://medium.com/@grepLeigh/real-time-object-tracking-with-tensorflow-raspberry-pi-and-pan-tilt-hat-2aeaef47e134

[Martin2kid]

Nomaan,

I've used pan & tilt face tracking using haarcascade_frotalface_detection method, it's detection accuracy is nowhere near TensorFlow alone approach as she accomplished here and made available.(only difference is person detection vs face detection)

Adrian also provided TensorFlow setup instruction in his post but without Pan & Tilt integration (He used Haarcascade method in his Pan & Tilt tracking tutorial)

Adrian also used Haarcascade method to crop face within TensorFlow in his 2 tutorial and here is 2 of 2 link for your reference; https://www.pyimagesearch.com/2019/04/22/getting-started-with-google-corals-tpu-usb-accelerator/