add an object tracker, or an Id for each object ?

[obendidi]

Hello, is there a way to add an object tracker to yolo using opencv ? Or something like giving an id to each object and tracking it's mouvement over the video ?

Thank you !

[jubjamie]

I've had a look at this problem and it's not very easy. The boxes are reported in order left to right I think. Yolo works frame by frame and doesn't distinguish between the objects. It only predicts areas where an object may be.

I had a quick go at object tracking with openCV yesterday but found it a bit rubbish. If performance isn't an issue then you could have some more complex methods to analyse the different predicted regions between the frames to match them up to the ones before but with the variance and instability of yolo this might not be very good. (Not to mention that it is a very slow way of doing things).

You could try looking at other object trackers and using yolo to provide the initial bounding box area and then handover to the object tracker but then you don't know if a new object has entered the frame unless you start comparing current tracked objects to yolo detections and trying to match them.

If anyone else has ideas on this I would also be keen to hear about them!

[obendidi]

I've also thought of using Yolo to detect the objects only on the starting frame and then handing things out to an object tracker in opencv, and then on each new frame only use Yolo to detect only boxes that are on the fringes of the Image (for new objects ) add them to the tracker and repeat.

but as you've said @jubjamie this is a very slow way of doing things, and not very practical Thank you

[jubjamie]

Yeah, I tried yesterday VERY quickly with OpenCV and half of the trackers crashed the program upon load (probably my fault tbh) but the ones that did work got lost quite quickly. Could possibly leave the trackers for 1 second then update all boxes with yolo but then you lose your box IDs.

I tried implementing a system that looked at yolo boxes between each frame and then if two were close to each other it assumed they were the same object. It was very basic but I couldn't really tell if it worked very well. I ended up scrapping it as I was still getting ID changes (and it wasn't critical). I ended up just having certain box colours for each type of class. I know that's not what you want to do but that's where I got to haha. If you have any other ideas leave them here so we can keep the conversation going!

Cheers

[obendidi]

I've found these links, it might be interesting : Issue Tracking Yolo C++ Issue 2 Tracking Yolo C++ issue tracking implemented in C++

so just need to understand the C++ code , convert and try it in python :)

[jubjamie]

Interesting. Let me know how you get on!

[Dhruv-Mohan]

I've made a very rudimentary implementation using c++ but the core idea is the same as what @jubjamie stated a few posts above.

Yolo is run asynchronously on a video stream. Whenever the yolo thread gets results the program compares the yolo given Bounding boxes to the bounding boxes of the objects being tracked.

You can then implement an algorithm to check if a Tracking bounding box and Yolo bounding box are enclosing the same object. I've experimented with template matching but i'm currently simply using Box overlap and distance from the centroids.

If the yolo bounding box is unable to find a corresponding tracking bounding box then the program assumes that the object enclosed by the yolo bounding box is new and initializes a tracker around it.

'KCF' trackers seem to work well for my use case.

[obendidi]

what is the fps , when you do the matching in each frame ? KCF trackers don't work well when the object is fully or partially hidden in some frames, I'm experimenting with the GOTURN tracker but I still didn't get the one implemented in opencv/caffe to work :( I've implemented one with tensorflow , but the results are not as good as they should be (by a far margin )

[Dhruv-Mohan]

When I match each frame on a 1080p video I get around 15fps. I could speed it up by gpu decoding the video and reducing the size of the frames that I feed into yolo .

You're right about KCF trackers performing poorly when an object is occluded. I've heard there's a bug with the implementation of GORTUN in opencv 3.2 maybe thats why it doesn't work.

[jcarletgo]

Not opencv but https://github.com/abewley/sort is pretty simple to use

[OmarMAmin]

i second @jcarletgo
sort is a multitarget tracking algorithm, which you can use to match trackers predictions with detections, it's based on Kalman Filter tracking, which doesn't track visual cues, instead it adjusts object speed and direction after each detection matching (observation), but you can very easily replace Kalman Filter with any other tracker that you find suitable to your application

Dlib has a very good and fast correlation tracker, you might want to check if it'll be more robust than Kalman Filter or not. http://dlib.net/video_tracking_ex.cpp.html

I hope this is useful, and let me know what you tried so far if that's possible

Thank you

[obendidi]

I'm currently trying with deep_sort : https://github.com/nwojke/deep_sort/issues/4 it seems to give better results, remain to see at what fps the tracking will be ...

[OmarMAmin]

dlib tracker is very fast, but fpr sure a deep learning based approach would be better

[obendidi]

here is my take on tracking using YOLO and deep_sort/sort (people tracking ) : https://github.com/bendidi/Tracking-with-darkflow

let me know possible improvements and changes ! thanks

[abhigoku10]

@Dhruv-Mohan can u pls share the algorithm flow or the code which u have implemented ???

[Dhruv-Mohan]

@abhigoku10 I'd suggest that you take a look at the sort repository that jcarletgo posted. It's leagues better than my own implementation of multi object tracking

[3073]

@bendidi: I have seen your repo on this and it is well organized. but I did not see the tracker and the detection together. that means the functions inside sort are not called in predict.py. why? if bounding box of the detector is used for initializing the tracker. sorry if I mistaken. thank you

[obendidi]

@3073 sort is actually imported here , it is not clean , but i'm still testing many other trackers and other object detectors (detectron, object Detection API ) , and will compile everything in one cleaner repo shortly

[3073]

Thank you Can we have a deal so that you teach me all this ''Detection plus Tracker code'' line by line and I donate you $100 for your time and explanation?

I am a student and I need to learn so that I can able to implement this tracker(SORT) for SSD detectors.

sorry if it is shame.

On Wed, Feb 21, 2018 at 1:27 AM, Ouail notifications@github.com wrote:

@3073 https://github.com/3073 sort is actually imported here https://github.com/bendidi/darkflow/blob/bc2f5c0116d7770c49d0e6662f6fc2d3dcd2a73f/darkflow/net/help.py#L84 , it is not clean , but i'm still testing many other trackers and other object detectors (detectron, object Detection API ) , and will compile everything in one cleaner repo shortly

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/thtrieu/darkflow/issues/326#issuecomment-367263758, or mute the thread https://github.com/notifications/unsubscribe-auth/AgXgdenak0JJau_I2xYZiGjXnDKrk2iPks5tW-GfgaJpZM4ONTgm .

[matriculus]

I tried to implement opencv trackers with tensorflow object detection api to detect and track object. But it has some issues.

[rachelspykerman]

@pradeeshvishnu how did you go about doing this? I recently got tensorflows object detection api working, and now want to implement object tracking, but I'm not sure how to go about starting on this. Ideally, I'd like to be able to differentiate between similar detected objects. For example, if I detect two people in one frame, I'd like to be able to track each person individually for future frames. Any pointers would be helpful :)

[3073]

I have implemented SSD detector with dlib tracker and it works but it is very very slow to the extent that it can not be used for real time applications

[matriculus]

@rachelspykerman Please have a look at this page https://www.learnopencv.com/object-tracking-using-opencv-cpp-python. There are different object trackers provided by openCV. What I did is that, I used tensorflow's api to detect the objects and then I got the box coordinates from the tensorflow function and used those boxes to initiate the tracker. OpenCV trackers need an initial area (a region using a box's coordinates) of interest and then it'll track the same region throughout the rest of the frames. So, the boxes drawn by the tensorflow api will help as a region of interest.

[matriculus]

You can look at my code here. For every 15 frames, the objects are detected in the 1st frame and in the next 14, the detected objects are tracked.

[3073]

thank you for that....I used SSD detector to initialize and dlib tracker and it works but it is too slow... how can make it fast? please help if any

On Thu, May 17, 2018 at 12:10 PM, Vishnu Pradeesh notifications@github.com wrote:

You can look at my code here https://github.com/pradeeshenator/Computer-Vision. For every 15 frames, the objects are detected in the 1st frame and in the next 14, the detected objects are tracked.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/thtrieu/darkflow/issues/326#issuecomment-389976300, or mute the thread https://github.com/notifications/unsubscribe-auth/AgXgdScNrqda3lNgO_0GizoU5eSHlzJ5ks5tzcsggaJpZM4ONTgm .

[nishantasarma]

@pradeeshenator will this code give the box coordiinates for multiple objects at once?

[matriculus]

Ya it'll give for multiple objects.

[andyrey]

There is own rather decent tracker in YOLO, thanks to AlexeyAB, looks like released on GPU:

std::vector<bbox_t> Detector::tracking(std::vector<bbox_t> cur_bbox_vec, int const > frames_story),

it simply translates objects id from one frame to the next, based on distance. If no hard occlution, works not bad. Do we need tracker for mitigate GPU or CPU load due to detection excluding? It has sense if tracker is easier than detector. If we apply that of from dlib- correlation tracker, as 3073 noticed, it is slower, bcz released on CPU only. __

[Rahul-Venugopal]

@andyrey Is there any resource where i can look into more details of this ? Thanks in advance