Can the experimental tracking module already be used?

[sandromario]

Issue Summary

There seems to be an experimental tracking module (for linking persons in following video frames to each other) in the code (master branch), but to the best of my knowledge it's not documented anywhere and I'm not sure whether it is functional already. It would be great if someone with good knowledge of the code could tell me if/when/how this feature can be used. Many thanks!

Type of Issue

• Question

Your System Configuration

not relevant

[gineshidalgo99]

(Edited) We're working on it, hopefully it should be released around Jul/Aug.

[sberryman]

@gineshidalgo99 any idea when this coming? Also, more importantly for me, will I be able to run just the tracking component against previously detected poses or do I need to run the full inference again?

In about 8 days my computers will be done processing roughly 7M frames of video using your maximum accuracy configuration. It would be a shame to waste all those resources if this is coming quickly.

I've previously run a sampling (2 FPS) of those same frames through at the default network size and manually placed the detected people from each frame into an identity/id spanning 60 wall-clock seconds.

Once I have all the frames for the 3 days (daylight hours only) processed at 30 FPS and maximum accuracy I would like to fill in the gaps. My plan is to implement OpenCV's tracker and use my known/identified poses as bounding boxes for the tracker. We're not talking about a small number of unique identities either, at least 1-2k per day with the avg person visible for 30-60 seconds.

[gineshidalgo99]

The current expected date is around Aug/Sep at this point. I'm sorry for this month delay, we are limited in resources.

Initially, you will have to run the tracking component with the full inference.

But given that it's a sub-module inside OP, you could also implement your own demo of the tracking version to read the previously generated JSON files as input, and perform the tracking on top of them (but there will not be a out-of-the-box function for this initially, you will have to use the C++ API for that).

[sberryman]

Wow, thanks for the quick response! No need to apologize on the timeline, when do tech timelines ever really work out? I just figured it wouldn't hurt to ask, since I know the true identities at 2 FPS I could help partially test it when you are ready.

How are you planning to implement the tracking (simple distances between joints)? What about occlusions (partial or full). I'm assuming you are NOT going down the route of ReID so you are creating tracklets and leave it to us to connect the tracks if occlusions occur?

[gineshidalgo99]

Tracking for now, Re-Id is in our very-long to-do list for now

[longfenghou]

Hello, openpose is really a great work. Concerning the tracking, which method are you planning to use?

[jbkarle]

Hello, amazing work!. Is tracking module still in the radar for Feb-Mar?

[gineshidalgo99]

(Edited) May-June is the expectation at this point, sorry!

[longfenghou]

Hello， I want to know which kind of method are you planning to use for implementing the tracking, is the imple distances between joints will work or you may have other better method, I have found a lot of tracking methods in the literature, and I am planning to implant on to your code myself, can I get some advise from you, thanks!

[gineshidalgo99]

LK tracking most probably (optical flow)

[longfenghou]

thanks for your recommendation, I will have a try on this method.

[longfenghou]

Hi, I want to analyse the behavior of students in the classromm (most of the time sit down) based on the key points given from the opepose, such as hands up, stand up, sleep (bored), is there any interesting method that can be applied for such kind of applications? According to my thought, I will need to first “identification”(using facial key points), then I will need the “tracking”(using total key points, is simple distance calculation possible? I found that LK method is based on the original image data, not the key points generated by openpose, this will complicate the things), finally, I will need to do the “behavior analysis”( hands up, stand up, sleep (bored),) according to the temporal key points data. It is a huge work to do all these three points, can you give me some advice. Thanks.

[GarrickLin]

the tracking module is out of experimental but still not working now ? raise exception here any further plan on this module ? @gineshidalgo99

[gineshidalgo99]

We have paused it for now given our priorities in a different problem, but it should be there in 2-4 months. You can actually try it if you disable that exception, but it only works for 1 person at the time (number_people_max 1). It doesn't work for multiple people as it requires first identification, which is unfinished yet.

[cmu-dev4realz]

OMG @gineshidalgo99 It's the end of August. You LIED to us??? How could you do this? We used your OpenPose library in good faith in our educational projects and now we have NO TRACKING. My thesis project is RUINED. I am going to tell all my friends not to attend CMU until this is fixed. Damn.

[sberryman]

Several good lessons here @cmu-dev4realz. First, you get what you pay for. In this case, I’m presuming you haven’t paid the developers to implement tracking. Second, relying on an unreleased feature for any project is extremely risky.

This is an open-source project maintained by individuals whose priorities are (most likely) dictated by the lab and/or licensee(s). It is quite obvious by the delay that tracking is not currently a priority. Tracking is a fairly trivial problem of which you can find several public domain examples. PoseFlow by AlphaPose has an example which you can modify to work with OpenPose.

Lastly, the definition of “lie” is

“a false statement made with deliberate intent to deceive; an intentional untruth; a falsehood.”

I would find it hard to believe and next to impossible to prove that @gineshidalgo99 intent was to deliberately deceive. You would be doing your friend(s) a disservice as CMU continually pushes state of the art in the AI/ML field. Attending CMU would provide an opportunity to work alongside some of the leading researchers in the field.

(I am in no way affiliated with CMU and my opinions regarding CMU are from a third party perspective based on reading arXiv preprints for the past year.)

[gineshidalgo99]

Thank you very much @sberryman for the moral support, it's highly appreciated.

About the tracking, I have released the current experimental version. Check the README, in particular the latest features section, where it shows a hyperlink to some examples. Best