Questions about the weights and inference

[indigo1973]

First of all, thank you very much for your contribution to this work, I have gained some methods and insprition from it. However, I still have a few questions. First, in the weight files you provided, is synthetic.pth the final weight trained on the synthetic domain? Using this weight directly, I got a score of 0.082 instead of the 0.039 as shown in TABLE III of your paper. However, after further training with the training set tag, my score improved to 0.024. Figure I. sythetic.pth and retraining weight inference on the validation domain

There should be no big error in the ESA score calculation, because using the weights you provided for sunlamp and lightbox domains, I can get 0.622 and 0.099 respectively. Here, I used USAC_MAGSAC for the PnP algorithm.

Secondly, no matter what kind of weight I get on the synthetic domain, I can’t get a score less than 0.5 when I apply it directly to the lightbox domain. Are there additional strategies for the inference phase not covered in the papers and code? I suspect there may be an issue with the inference process, potentially leading to overfitting of the pseudo-labeling noise during iterations, which has hindered my ability to reproduce the self-training process and achieve the best results.

[JotaBravo]

Hi,

Thanks for the interest in our work!

1) yes, synthetic.pth is trained on the synthetic domain.

2) the inference strategies are in utils.py. As we mention in the paper/code we employ EPnP with a confidende of 0.99 and a reprojection error of 2.0. Make sure those match. We also filter the keypoints, by retaining the 7 strongest keypoints.

3) I'm not sure I understand your last question. Training on synthetic and testing on the HIL domain (lightbox) and achieving 0.5 is quite a good performance in my experience. If you perform the pseudo-labelling iterations reported in the paper (e.g. extracting pseudo-labels and training) you should get similar performance to the one reported in the paper. Remember that that process is coded into utils.py

[indigo1973]

Dear Bravo: Thank you so much for your reply, I still have some questions to confirm. Were the scores in the last row of Table 2 and Table 3 obtained using the synthetic.pth weight file? Figure I. Screenshot from your paper “Spacecraft Pose Estimation: Robust 2D and 3D-Structural Losses and Unsupervised Domain Adaptation by Inter-Model Consensus”

When I tested this weight file on the sunlamp and lightbox domains, I got 1.056 and 1.01, respectively. This is quite different from the data in the table above. Figure II. Synthetic.pth inference results on three domains