Closed cygbbhx closed 3 months ago
Thank you for your interest in our work. For your questions:
Thank you for your kind response! To ensure I understand correctly, I have two more questions I'd like to clarify:
I understand that the target embeddings can be parallelized. According to the paper, the historical trajectory includes both target embeddings and ID embeddings. Then, is it correct to assume that ID embedding extraction is not parallelized as the ID Decoder requires ID embeddings from previous trajectories? Despite this, since DETR output embeddings can be processed in parallel, does this mean that the ID Decoder's processing time is relatively shorter than DETR's, thereby still allowing for an overall speed-up in the process?
Regarding sparser training, then would updating DETR with every frame enhance the overall performance? I'm curious if previous related works typically relied heavily on frequent updates of DETR for optimal performance.
I see! Now everything makes sense. Thank you, I appreciate your detailed response.
Hello, I read your paper and have several questions.
My question is about Section 4.2 / Appendix D - parallelized training. I think I haven't fully understood why the use of ID Decoder enables parallelized training.
From my understanding, ID Decoder computes the prediction using Historical Trajectory information (and current detection token). Doesn't the historical trajectory still require serial processing?
In the paper, it is noted that only 4 frames are used for gradient recording and others are passed through gradient-free mode. In Appendix D, I see 2 frames are processed with grad out of 5. How are these numbers (number of frames with grad/number of forward passes) calculated?
Related to Question 2, why are only few frames selected with gradients? Does this mean the ID Decoder enables more robust learning, which allows sparser training of the detector?
Thank you for your great work and contribution!