Open AlexeyAB opened 5 years ago
thanks for sharing @AlexeyAB . How can we use this, with yolo detection. secodly, how different is it to already exisiting (OPTFLOW AND KALMAN FILTERING)
It's really great you are working on baking this into your YOLO fork, I had good results using a very simple one: https://github.com/bochinski/iou-tracker , paper: http://elvera.nue.tu-berlin.de/files/1547Bochinski2018.pdf
I implemented a flaky node.js version: https://github.com/tdurand/node-moving-things-tracker
It's really great you are working on baking this into your YOLO fork, I had good results using a very simple one: https://github.com/bochinski/iou-tracker , paper: http://elvera.nue.tu-berlin.de/files/1547Bochinski2018.pdf
I implemented a flaky node.js version: https://github.com/tdurand/node-moving-things-tracker
Had good experience too. Overall, it's ultra-fast!
Hi! I and Smorodov working with multi-object tracker: https://github.com/Smorodov/Multitarget-tracker We use your fork as one of detector. And if you do not mind, then I could transfer part of the code to this project.
@Nuzhny007 Hi,
What is the most accurate multi-object tracker you could find and used?
I'm working only with C++ code with real time systems. And Hungarian algorithm or version with bipartite graph is not bad. And also i'm used more simple algorithms but they are not robust.
A good online algorithm is "FollowMe: Efficient Min-cost Flow Tracking with Bounded Memory and Computation": http://www.cvlibs.net/projects/online_tracking/ But it only on Python. It's implementation on C++ in my plans.
The good are "Lifted Multicut" (https://github.com/jutanke/cabbage) and LBM (https://github.com/Smorodov/Multitarget-tracker/issues/125) but they are too slow.
And also in plans SORT with YOLO: https://github.com/humoncy/YOLOv3-SORT-ReID
@Nuzhny007
Yes, it seems that SORT is very fast and enough accurate algorithm.
I'm working only with C++ code with real time systems. And Hungarian algorithm or version with bipartite graph is not bad.
What is the different between your Hungarian algorithm and common SORT-algorithm (cv::KalmanFilter + Hungarian algorithm) ?
Did you try to combine: cv::cuda::SparsePyrLKOpticalFlow
+ cv::KalmanFilter
+ Hungarian algorithm ?
Did you try cv::TrackerCSRT()
(CSRDCF++) from OpenCV-contirb?
For example ( https://github.com/Smorodov/Multitarget-tracker/blob/master/example/examples.h#L177 ):
TrackerSettings settings;
settings.SetDistance(tracking::DistJaccard);
settings.m_kalmanType = tracking::KalmanUnscented;
settings.m_filterGoal = tracking::FilterRect;
settings.m_lostTrackType = tracking::TrackCSRT;
settings.m_matchType = tracking::MatchHungrian;
Here we use SORT: DistJaccard + Kalman + Hungarian. And we use CSRT tracker (TrackCSRT) for frames when object was undetected.
On image trajectory parts with circles means that here was used visual object tracker (KCF, CSRT, STAPLE or...)
@Nuzhny007
So circles means, that the SSD-detector can't detect object there, and was used object tracker (KCF, CSRT, STAPLE or...).
Is it this video? https://www.youtube.com/watch?v=Qssz6tVGoOc It seems that
Why SORT (Kalman + Hungarian) and CSRT can't solve this taks, even if these objects are clearly visible and almost do not overlap?
Here was used not CSRT and classical KCF from OpenCV - it more faster but not so robust as CSRT. And KCF not robust for dynamic object like people, it tracks static background.
@Nuzhny007
Hi! I and Smorodov working with multi-object tracker: https://github.com/Smorodov/Multitarget-tracker We use your fork as one of detector. And if you do not mind, then I could transfer part of the code to this project.
Yes, you can transfer tracking part to this project by using Pull Requests
to darknet https://github.com/AlexeyAB/darknet/blob/master/src/image_opencv.cpp
or/and to ./uselib
project that uses libdarknet.so
- files https://github.com/AlexeyAB/darknet/blob/master/src/yolo_console_dll.cpp and https://github.com/AlexeyAB/darknet/blob/master/include/yolo_v2_class.hpp
I think there can be added some #ifdef
to check if we OpenCV is compiled with OpenCV-contrib (KCF, CSRT, STAPLE or...).
or can be added #define
in https://github.com/AlexeyAB/darknet/blob/master/Makefile#L9 for example, TRACK_OBJECTS=1
to compile with Tracking functions. We should set this flag TRACK_OBJECTS=1
if we compiled OpenCV with OpenCV-contrib.
Ok, I'l do it
@Nuzhny007 Hi,
Thanks for your work. I have merged this repo with your repo. I'm going to use for traffic sign detection and my camera is front of the car. Sometimes the signs are not recognized in some frames. I want to know, what is the best TrackerSettings for my case? Is TrackCSRT better than TrackKCF for my case?
Thanks
@AlexeyAB What is the status of multi-object tracking in this repo? Was @Nuzhny007 tracker ever integrated?
@LukeAI Not started yet.
We have one problem: Multitarget-tracker has GPL license and I can't use it code in darknet. Smorodov (the owner of the Multitarget-tracker) starded a new project TrackerGenerator for generate tracker's code with new license.
@zpmmehrdad CSRT tracker is better.
If anyone is interested, I implemented an optimized (real-time) tracker based on Deep SORT that uses Darknet's YOLO models converted to TensorRT: https://github.com/GeekAlexis/FastMOT
@GeekAlexis Great, thanks!
yolov4-p6.cfg/weights
and DeepSort. Does it also supports yolov4-csp-x-swish.cfg/weights
(it just uses Swish-activation)? https://github.com/AlexeyAB/darknet#pre-trained-models
- yolov4-csp-x-swish.cfg - 640x640 - 69.9% mAP@0.5 (51.5% AP@0.5:0.95) - 23(R) FPS / 50(V) FPS - 221 BFlops (110 FMA) - 381 MB: yolov4-csp-x-swish.weights
- pre-trained weights for training: https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v4_pre/yolov4-csp-x-swish.conv.192
Did you try to use SiamRPN++
from OpenCV, and/or compare DeepSort vs SiamRPN++? https://github.com/opencv/opencv/blame/master/samples/dnn/siamrpnpp.py#L6-L13
Quite challenging case at 5:47 https://youtu.be/PdZrF145lAI?t=348
Also just wondering how does this compare to FairMONT? https://github.com/ifzhang/FairMOT
@AlexeyAB
Thanks for your interest. I will support swish CSP weights.
I have not tried to use SiamRPN++. Because my implementation uses KLT (optical flow) with DeepSORT, a possibility is to replace KLT with SiamRPN++. It should improve accuracy but a single object tracker probably doesn't scale well with the number of objects in a real-time MOT system.
FairMOT is a faster extension of Deep SORT because it combines detection and ReID embedding in one network. I have plans to support it if I can convert its YOLOv5 backbone version to TensorRT. The author's implementation is not as optimized and its tracking framework is not as robust to camera motion and occlusion compared to mine.
Also, combining FairMOT with a scaled-YOLOv4 backbone would be nice.
Add multi-object tracker (dynamic array of single object trackers):
SORT-tracker:
cv::KalmanFilter
+ Hungarian algorithm https://en.wikipedia.org/wiki/Hungarian_algorithm and https://arxiv.org/abs/1602.00763 and https://github.com/mcximing/sort-cppcv::TrackerCSRT()
(CSRDCF++) - OpenCV-contirbObject detection task and appearance embedding task simutaneously in a shared neural network - source: https://github.com/Zhongdao/Towards-Realtime-MOT pdf: https://arxiv.org/abs/1909.12605v1
SiamRPN - first places in Real-time: https://arxiv.org/abs/1808.06048 and https://github.com/foolwood/DaSiamRPN
Accuracy:
Page 16: https://prints.vicos.si/publications/files/365