search

weigao95 / surfelwarp

SurfelWarp: Efficient Non-Volumetric Dynamic Reconstruction
https://sites.google.com/view/surfelwarp/home
BSD 3-Clause "New" or "Revised" License
276 stars 71 forks source link

About the jacobian. Why don't calculate the full derivative from cost function to each node's twist? #58

Closed xiayeqingfeng closed 3 years ago

xiayeqingfeng commented 3 years ago

Hi @weigao95, thank you so much for sharing this well-implemented algorithm.

When reading the codes in detail, the jacobian calculation process immediately catch my attention.

As for the data term(dense depth term) in your paper, I notice that you calculate the jacobian from scalar cost to the warpped vertex's SE(3), and then map this jacobian to the node's twist with a weight. I wonder whether this is the right way to get the jacobian for each node's twist that we want optmize.

For myself, I will first build the jacobian directly from scalar cost to the node's twist, then build the JtJ and Jte matrix. Thus the building process of term2node and node2term structure in your code can be saved(I guess).

In conclude, I have the following two questions (take the dense depth term as example):

  1. Is it ok to optimize the node's twist by using the jacobian from scalar cost to the warpped vertex, plus a weight? does this process has a theoretic foundation?
  2. What is your consideration by deciding not to calculate the full derivative from scalar cost to the node's twist?

Any discussion is welcome! Thanks in advance!

weigao95 commented 3 years ago
  1. It is theoretically correct. I remembered that I've written a derivation in my FilterReg paper, and you might take a look.
  2. Because the twist jacobian can be easily computed for all 3 components. There is no need to mess with scalar costs.
xiayeqingfeng commented 3 years ago
  1. It is theoretically correct. I remembered that I've written a derivation in my FilterReg paper, and you might take a look.
  2. Because the twist jacobian can be easily computed for all 3 components. There is no need to mess with scalar costs.

Thanks for your reply! I will read your FilterReg paper for the derivative process asap. For now I'll leave this issue open in case I have further questions about the jacobian :) This issue will be closed once I fully understand the related problem.~

xiayeqingfeng commented 3 years ago
  1. It is theoretically correct. I remembered that I've written a derivation in my FilterReg paper, and you might take a look.
  2. Because the twist jacobian can be easily computed for all 3 components. There is no need to mess with scalar costs.

Hi @weigao95, For Question 1, I've read your FilterReg paper supplemental material. But I still can't understand how the computation of derivative from final cost to node's twist is avoided.

In section 3 of your supplemental material, you clearly mention that we need to get the Jacobian from residual to node's twist so that the final A matrix can be formed(Red rectangular area in the following pictures). screenshot-20210207-151840 screenshot-20210207-152426

In your code, however, the Jacobian(J1) from residual to nodes' twist is replaced by the Jacobian(J2) from residual to warpped vertex(which is much easier to compute). From my understanding, J1 should at least involves some dual-quaternion derivatives since dual quaternion blending is used.

Do you just approximate J1 with J2? If so, does this approximation have theoretical foundation?

Thanks again :)

weigao95 commented 3 years ago

It is not avoided. You might talk a look at geometry_icp_jacobian.cuh, which is used to compute the twist jacobian. There is no approximation here and I don't get what your J2 means.

If you think you can avoid term2node or node2term indexing, you are very welcomed to code it and we can discuss based on that.

Chang-Che-Kuei commented 2 years ago

Hi @xiayeqingfeng , have you solved your problems? May I ask you some questions via the email? Try not to mess this issue thread.