How to estimate the deformation residual and correspondence residual?

[ZJU-PLP]

@JiazeWang Hi, Jiaze:

In your paper III method, section D. Recurrent Reconstruction Network, you update the deformation residual by D1 = Dgt−D0 and correspondence residual by M¯ 1 = (M0)−1 × Mgt. However, I cannot find the details about Dgt and Mgt in your released code. I am confused that how can you get the Mi = Mi−1 × ¯Mi while M¯ i is not known ?

Could you mind sharing the method or details to get Mgt? Just showed as beloew:

[SocialGoodAI]

Hi, Linpeng:

In this paper, M{gt} refers to the ground truth label of the correspondence matrix, and D{gt} refers to the ground truth label of the deformation field. Reconstruction Loss, Correspondence Loss, and Recurrent Loss are used to supervise the learning process.

[ZJU-PLP]

@JeremyWANGJZ Thanks for your reply.

However, I cannot find the code of M_{gt} refers to the ground truth label of the correspondence matrix in your network.py. Specifically, I cannot understand why there is not the corresponding code of implementing format (3). Could you mind telling me where is the code about format (3) of M(hat)^i?

[SocialGoodAI]

Hi, we do not use the ground truth label of the correspondence matrix to directly supervise the learning process, we use Reconstruction Loss, Correspondence Loss, and Recurrent Loss to estimate the ground truth instead, thus the M_{gt} is not directly shown in our code.

The corresponding code of implementing format (3): https://github.com/JeremyWANGJZ/Category-6D-Pose/blob/2b8a380ed2630a56b152316010ed18e9ee1c58f7/lib/network.py#L191, L192, L209 and L210.

[SocialGoodAI]

Thanks for your interest in our work. Feel free to contact me if you have further questions.

A kind reminder: you may also refer to SGPA (https://github.com/leo94-hk/SGPA), an ICCV oral paper that achieves better results and uses less training time.

[ZJU-PLP]

Hi, we do not use the ground truth label of the correspondence matrix to directly supervise the learning process, we use Reconstruction Loss, Correspondence Loss, and Recurrent Loss to estimate the ground truth instead, thus the M_{gt} is not directly shown in our code.

The corresponding code of implementing format (3):

https://github.com/JeremyWANGJZ/Category-6D-Pose/blob/2b8a380ed2630a56b152316010ed18e9ee1c58f7/lib/network.py#L191

, L192, L209 and L210.

@JeremyWANGJZ Okk. I see. Thanks for your patient reply. I will follow your nice work SGPA.

Another question, could you mind illustrating how to understand index in here and how to use index? I cannot understand the usage of index.

network.py L163-L164

index = cat_id + torch.arange(bs, dtype=torch.long).cuda() * self.n_cat
assign_mat = torch.index_select(assign_mat, 0, index)