When object is out of FOV & re-appear, Rpi object tracking is not resuming

[Martin2kid]

• rpi-deep-pantilt version: 1.2
• Python version: 3.7.3
• TensorFlow version: 2.2
• Operating System: Buster 8-20-2020

Description

Describe what you were trying to get done; I tried to make RDP track face continueously with Edge TPU on Pi4 8GB with clean install only with RDP and I expected tracking would resume when subject re-appear in FOV but it hangs.

Tell us what happened, what went wrong, and what you expected to happen.; When I run (.venv) pi@raspberrypi:~ $ rpi-deep-pantilt track --edge-tpu face, As long as subject is in FOV, it works but when subject goes out of FOV, within in 10 second, cam is pointing up toward ceiling or facing down and stuck in there -- no tracking resume

When I run with (.venv) pi@raspberrypi:~ $ rpi-deep-pantilt track --edge-tpu person, As long as subject is in FOV, it works but when subject goes out of FOV, cam remain toward position of last jitter up & down and when subject return within FOV, cam is stuck pointing toward last position & stuck in there. -- no tracking resume

Paste the command(s) you ran and the output.
(.venv) pi@raspberrypi:~ $ rpi-deep-pantilt track --edge-tpu face
(.venv) pi@raspberrypi:~ $ rpi-deep-pantilt track --edge-tpu person

If there was a crash, please include the traceback here.

No Crash but, Upon use of face label; (.venv) pi@raspberrypi:~ $ rpi-deep-pantilt track --edge-tpu face ^C[INFO] You pressed ctrl + c! Exiting... [INFO] You pressed ctrl + c! Exiting... Process Process-2: [INFO] You pressed ctrl + c! Exiting... Traceback (most recent call last): File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() File "/usr/lib/python3.7/multiprocessing/process.py", line 99, in run self._target(*self._args, **self._kwargs) File "/home/pi/.venv/lib/python3.7/site-packages/rpi_deep_pantilt/detect/camera.py", line 38, in run_pantilt_detect prediction = model.predict(frame) File "/home/pi/.venv/lib/python3.7/site-packages/rpi_deep_pantilt/detect/facessd_mobilenet_v2.py", line 139, in predict input_tensor = input_tensor[tf.newaxis, ...] File "/home/pi/.venv/lib/python3.7/site-packages/tensorflow/python/ops/array_ops.py", line 944, in _slice_helper new_axis_mask |= (1 << index) KeyboardInterrupt

Aborted! ^CError in atexit._run_exitfuncs: Traceback (most recent call last): File "/usr/lib/python3.7/multiprocessing/popen_fork.py", line 28, in poll pid, sts = os.waitpid(self.pid, flag) KeyboardInterrupt

Upon use of Person label; (.venv) pi@raspberrypi:~ $ rpi-deep-pantilt track --edge-tpu person ^C[INFO] You pressed ctrl + c! Exiting... [INFO] You pressed ctrl + c! Exiting... [INFO] You pressed ctrl + c! Exiting...

Aborted! Process Process-2: Traceback (most recent call last): File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() File "/usr/lib/python3.7/multiprocessing/process.py", line 99, in run self._target(*self._args, **self._kwargs) File "/home/pi/.venv/lib/python3.7/site-packages/rpi_deep_pantilt/detect/camera.py", line 38, in run_pantilt_detect prediction = model.predict(frame) File "/home/pi/.venv/lib/python3.7/site-packages/rpi_deep_pantilt/detect/ssd_mobilenet_v3_coco.py", line 160, in predict self.tflite_interpreter.invoke() File "/home/pi/.venv/lib/python3.7/site-packages/tflite_runtime/interpreter.py", line 506, in invoke self._interpreter.Invoke() File "/home/pi/.venv/lib/python3.7/site-packages/tflite_runtime/interpreter_wrapper.py", line 118, in Invoke return _interpreter_wrapper.InterpreterWrapper_Invoke(self) KeyboardInterrupt ^CError in atexit._run_exitfuncs: Traceback (most recent call last): File "/usr/lib/python3.7/multiprocessing/popen_fork.py", line 28, in poll pid, sts = os.waitpid(self.pid, flag) KeyboardInterrupt

Am I missing any component or applying wrong command?

[leigh-johnson]

Thank you for reporting this @Martin2kid, sorry about the behavior here being difficult to debug. I'm guessing the default PID gains do not work for the distance or relative angle of the camera to tracking target in your setup. Another possibility is that your camera needs to be rotated (--rotation <int>). The default rotation values match the camera orientation in this picture:

In 2.0 (target: end of year), I'm shipping a rework of the PID control system that will fix this behavior for the face model. I'm going to close this and consolidate updates about the PID system in #48.

I'm truly amazed at all of the prototypes being developed from this quick tutorial! The original tutorial code has poor API surfaces for additional development though, so I'm rewriting this tool to be usable as a library. I want to make it dead-simple to generate a new prototype app, optionally bring your own model, and calibrate the control scheme.

[Martin2kid]

Leigh,

Thank you much for input! I know my camera setup & orientation is exactly same with yours but about 3 to 8' distance.

I also suspect new Buster release may have something to do with this behavior (Your setup, I believe is started from pre-Buster 8-20-2020 release vs mine is Pi4 4GB with new Buster release)

I'll try with diffirent PID tinkering & hope that'll fix this behavior in my setup.

BTW; Your setup & servo movements are much smoother and predictably solid than Intels's NCS2 with OpenVINO + SSD model setup(Quite bit of jerks & sudden jolt of occasional movement--occasional jitter and PID setting always goes into oscillation eventually, no matter what value I change and still work in progress)

Your work is superior & solid!!!

LMK, if you want to try Pablo's FOC (Dual Brushless motor FOC controller for closed loop position control--I ordered 2 and I have 1 extra for you to experiment with and share with us if you like to, I can also order 2 motor with encoder plus build a pan & tilt frame for you.

I believe, You can do bunch of cool control with it, dynamic PID setup, almost realtime tracking!