About the application of this module to other project

[duytrangiale]

Hi Jiayi,

Thanks for sharing this amazing work. I have read through the paper and can understand the main idea of your method. However, after going the example codes, I still don't know how to apply the RPMG to my own project.

Could you please describe the general steps I should do to use the RPMG layer in my network?

For my work, the output of neural network are rotation vectors that I then use it to compute the predicted rotation. My orientation loss is the difference between ground truth and predicted quaternions (which I convert from the rotation vectors).

Thanks for your help.

Cheers, Duy

[JYChen18]

Thanks for your attention to our work and sorry for the late reply.

To use our RPMG layer, you need to first convert the ground truth quaternion to a 3x3 rotation matrix. You also need to add the RPMG layer after the network prediction as in this line. Then, compute the loss between gt and the output of the RPMG layer, and you are done.

Jiayi Chen

[duytrangiale]

Hi Jiayi,

Thanks for your response. I will try again following your suggestion. I also have a few issues during implement your method, hope you can help me answer them.

1. When using your method, is it compulsory to compute the orientation loss as the difference between two rotation matrices? If not, can I compute this loss as the difference between two unit quaternions?
2. When I compute the loss using two rotation matrices, my network produces NaN values and I don't know why it does that. If I use quaternion, there will be no problem.
3. Can you please explain the step that you apply the Riemanian optimization on the loss to compute the goal rotation Rg. Is this a compulsory step? What if I already have ground truth rotation?

In my particular case, the network produces an intermediate rotation vector (let call it delta_r), I then apply this delta_r to rotate another orientation vector (r0) to produce the predicted vector (pred_r). This predicted vector will be compared with a ground truth rotation vector (gt_r) in the loss function.

Do you think my case is suitable to apply your method?

Sorry for asking too many questions and thanks for your help.

Cheers, Duy

[JYChen18]

Hi Duy,

1. You can compute loss between two unit quaternions, but you may have to implement it by yourself. You can only use the manifold mapping and not use the rotation mapping in our paper.
2. NaN is often caused by the transformation between different rotation representations, especially for SVD and division.
3. The key of our method is to compute a R_g by Riemannian optimization. The motivation is R_g may be a better goal for a non-Euclidean space, since it is closer to the current prediction. If you directly use the ground truth rotation, it is just the classic way to compute loss.
4. As for your scenario, I am not sure but maybe you can take a look at this paper

By the way, since currently I am not studying in this field, I may not be able to give you valuable suggestions.

Jiayi

[duytrangiale]

Hi Jiayi,

Thanks a lot for your suggestion. I have a bit clearer idea about your approach, will need to do some more experiments to have better understanding.

I may come back here and ask you if I get stuck at some points in the future but hopefully it can work well.

Cheers, Duy