Kalman Filter and Optical Flow

fvlntn commented 6 years ago

Hello Alexey,

First I would like to thanks you for the very good work and support you're providing us everyday on this fork.

I trained a model to recognize hand gestures in real-time. This model is doing great, however since I have a low-end GPU I have approx. 9 FPS on Yolo V2 416*416 and ./darknet detector demo for instance use #define FRAMES 3 which adds latency. (even if FRAMES = 1) (I was using some Python YOLO wrapper to code faster that decreased framerate too). In order to have the best framerate possible, I decided to use your latest commit and use LIBSO=1 and ./uselib.

Since I want to do real-time tracking, I wanted to add Kalman Filter or Optical Flow Tracking myself but I saw you did it before me. I saw on the code that you defined Optical Flow so I used #define TRACK_OPTFLOW with OpenCV 3.4.0 and I obtained 18 FPS and overall less latency.

However I saw in yolo_console_dll.cpp that you defined track_kalman class, is there any way to use it yet?

AlexeyAB commented 6 years ago

Hi @ralek67

I have not tested yet Kalman Filter well, so I can't recommend it to use, it also works poorly in my tasks, when the camera chaotically moves - it increases the delay of the bounded boxes - detection lags behind the object.

Try to disable saving video, will FPS be increased? bool const save_output_videofile = false; https://github.com/AlexeyAB/darknet/blob/df076653e00db69c6fb57869981f5196a8f55e70/src/yolo_console_dll.cpp#L244
You can try to improve accuracy of Optical Flow increasing window size from 9 to 21 or more Tracker_optflow tracker_flow(21);: https://github.com/AlexeyAB/darknet/blob/df076653e00db69c6fb57869981f5196a8f55e70/src/yolo_console_dll.cpp#L246
And you can try to increase speed of Optical Flow by decreasing speed of Yolo, just add here these lines instead this 1 line: https://github.com/AlexeyAB/darknet/blob/df076653e00db69c6fb57869981f5196a8f55e70/src/yolo_console_dll.cpp#L365
```
using namespace std::chrono_literals;
std::this_thread::sleep_for(100ms);
++fps_det_counter;
```

fvlntn commented 6 years ago

Thanks for the tips for optical flow.

I saw you recommend window size to be (9 9) (15 15) (21 21) (31 31) in yolo_v2_class.hpp, where does that come from? I know it should decrease performance since window size is higher, but it doesn't seem to drastically lower my framerate when I went from 9 to 31 or even higher.

Even if Kalman Filter doesn't work that well, is there an example available to try it and see if it can be enhanced?

AlexeyAB / darknet

Kalman Filter and Optical Flow #460