zhen-he
commented
4 years ago Hi, thanks for your interest. I think the following mechanisms are important for the model to track objects accurately:
- Attention mechanism. Since the receptive field (RF) of each tracker is designed to be a local region on the image (via FCN + NTM, please see the last paragraph in Section 3.1 and "Model Configuration" in appendix A.1), each tracker can only (at most) reconstruct a local region (corresponding to the RF) rather than any part of the image (as each tracker cannot access the information out of its RF, it cannot reconstruct the region out of its RF).
- Tightness constraint. This avoids loose bounding boxes (please see Section 2.4).
- Parameter sharing for different trackers (regularization effect). As the parameters are shared by different trackers, these trackers must have the same behavior---if one tracker just learned to reconstruct arbitrary local region (which might contain no objects), then other trackers would also learn to do this, indicating a high reconstrution loss. Thus, only if each tracker learns to track one object a time and then erase the content of this object from the memory (please see Figure 8), can these trackers well-reconstruct the input image.
StudentWong
Hello, I have run the module and it has remarkable performance. But I am confused about the incentive for the loss to decrease. If the "Feature Extractor" and the "Tracker Array" was trained to a set of parameter in which they always perform Identical output representing any part of the input image, the reconstructed frame rendered according to these output apperance and pose can also be practically equal to the original frame.
Is there any mechanism in the module or in the RAT that motivates the "Feature Extractor" and the "Tracker Array" to output the exact tracking object since it's totally "label-free"?
Many Thanks and Best Regards