search

tomasjakab / imm

Code for unsupervised learning of object landmarks as proposed in "Unsupervised Learning of Object Landmarks through Conditional Image Generation", Tomas Jakab*, Ankush Gupta*, Hakan Bilen, Andrea Vedaldi, Advances in Neural Information Processing Systems (NeurIPS) 2018.
149 stars 25 forks source link

Can this work handle occlusion? #15

Closed towardsautonomy closed 3 years ago

towardsautonomy commented 3 years ago

Hello, I first want to thank the authors for this work. I just wanted to check if and how can we use this work for occluded or partially out of frames objects?

ankush-me commented 3 years ago

@towardsautonomy -- thanks for your message. While the method always predicts a keypoint even if the object is occluded (fully / partially), i.e., it does not handle occlusions explicitly, however, looking at the underlying heatmaps (esp. their peakiness) can inform how confident the model is about the location prediction, i.e., for occluded objects/parts the heat maps should be spread out. Hope this helps.