search

rawalkhirodkar / MIPNet

Apache License 2.0
74 stars 13 forks source link

Questions regarding the code and application #2

Closed poornimajd closed 3 years ago

poornimajd commented 3 years ago

Hello,just checked out your paper.Great work! I had the following questions - 1)Is there a approximate deadline for the code with instructions to be released for training and testing the model for our custom use-case? 2)The annotation for the ground truth generation is just the 2d key points of the particular object right,the network will automatically convert it to heatmap or do we have to give it as a heatmap? 3)Can we apply this to a case as follows- A camera is observing 2-3 moving robots in its FOV,and if I retrain the network with the images of the robot,with its corresponding ground-truth keypoints,the network can still predict the 2d keypoints of the moving robot right? 4)For this does the robot have to be at a particular distance from the camera,so that the keypoint estimation is accurate enough?Meaning is the network's accuracy dependent on "the distance the object is from the camera"?

Any suggestions/replies are greatly appreciated! Thank you

rawalkhirodkar commented 3 years ago

Thank you for your interest.

  1. We will release the training and testing instructions from the paper along with models weights end of November (after the CVPR 2022 deadline)
  2. We have dataset scripts which convert the 2d keypoints into heatmap. Please refer, lib/datasets/coco.py.
  3. Yes, this is an interesting application. You can train the MIPNet for robot pose estimation under occlusion.
  4. If you have training images which capture the scenario when the robot is close to the camera, it should work. The key is to have good training data.

Hope this helps.

poornimajd commented 3 years ago

Thank you for the quick and detailed reply! Awaiting the train test code instruction release,will check out the scripts.