Joint or separate training

[gurkirt]

Nice work! Thank you for sharing the code.

Is training of detector and tracker is joint or separate? It seems from the paper (Section 5.2) that the first detector needs to be trained then the detector is frozen and the tracker is finetuned after that? Is that right inference?

Thanks Gurkirt

[xingyizhou]

Hi,

Thank you for your interest. You are correct. Let me supplement more details. Both our TAO and MOT experiments train the detector first, and then add the tracking transformer layers and fine-tune. The MOT models train all layers jointly, and the TAO models freeze the backbone layers (but still train the FPN and detection heads jointly) during tracker fine-tuning.

Best, Xingyi

[xingyizhou]

Closing due to lack of activity. Feel free to reopen.