search

facebookresearch / DetectAndTrack

The implementation of an algorithm presented in the CVPR18 paper: "Detect-and-Track: Efficient Pose Estimation in Videos"
Apache License 2.0
1k stars 186 forks source link

Is this possible? #45

Closed codergg4393 closed 6 years ago

codergg4393 commented 6 years ago

Hey,

This is more of a question than an issue. Feel free to delete this if it's not the right place. I want to know, how can I actually detect and track a specific part of the body, say a tshirt and be able to add something on top of it.

So for example, adding a logo (image) on actor's tshirt in such a way where it looks like the actor is actually wearing a shirt with the logo on it (so the logo image actually moves as the actor moves in the video frames).

Is this possible? If not, please point me in the right direction.

Thanks in advance!

rohitgirdhar commented 6 years ago

This code can help you track the keypoints of the person, so you would be able to track the torso, legs and so on. You could use that information to overlay stuff onto the video such that it moves with the person. You could also use the 4 torso keypoints (left/right shoulder/hip) to transform the logo such that it looks like it's on the t-shirt.

codergg4393 commented 6 years ago

This code can help you track the keypoints of the person, so you would be able to track the torso, legs and so on. You could use that information to overlay stuff onto the video such that it moves with the person. You could also use the 4 torso keypoints (left/right shoulder/hip) to transform the logo such that it looks like it's on the t-shirt.

That's excellent. Couple quick questions though.

1) You said I will need to overlay stuff, so the code for overlaying stuff is included? Or I'll need to find that elsewhere?

2) Also, you mentioned using 4 key points to transform the logo such that it looks like it's on the t-shirt. Is the code for transformation included as well?

3) Does this work only for human bodies? Is there a way for me to add a logo on top of say a coffee mug that's on a video? Of course, the coffee mug might turn in the video so would it be able to track the points where the logo actually sticks to a certain point on the mug and move along with the mug?

Would really appreciate if you point me in the right direction in for getting this done.

Thanks again for your time and great work here!

rohitgirdhar commented 6 years ago
  1. It's not.
  2. It's not. You might want to look for homography transformations.
  3. This was trained for human bodies.