Calculate the spectrum

[YanhaoZhang]

Thanks a lot for this amazing work. I notice that the spectrum is provided. May I know how did you calculate it? I plan to train the network using my own data.

Another quick question is that I am wondering if the calculation of spectrum is differentiable? Thanks a lot.

[riccardomarin]

Hi @YanhaoZhang , I am glad that you find the work interesting!

The spectrum is computed with the implementation of LBO at third-order FEM. The third order produces more stable results, but I think you can start with the first order with any kind of implementation (I suggest the robust_laplacian library in python, which is particularly handy). If that does not work well (i.e., the spectrum are unstable for the kind of data\application you have in mind), I will ask my colleague if I can forward to you the code for the FEM3 version (I think there are no problem to do this).

The differentiation of spectrum computation is something that bother us for a long time :) yes, it can be done, but it is computationally expensive, particularly because the auto-diff libraries do not efficiently handle sparse matrices and truncated eigendecomposition. But it is also true that we questioned this a couple of years ago, and I do not know if things are changed now. If you are interested in this, I may link you to some of my colleagues more expert than me.

[YanhaoZhang]

Hi @riccardomarin Thanks a lot for your prompt reply. Your answer is really helpful for me.

I will try the robust_laplacian first as recommended. Do you mean this package here?

Regarding to the differentiation of spectrum, since it is not an easy problem, I will use the similar way as your paper, i.e. using a network to calculate the spectrum. One question I am not quite sure is that, is the network able to calculate correct spectrum is my testing mesh is very different from my training meshes? Suppose I train the network using meshes of head, and testing it with a sphere. Will the network able to predict correct spectrum of the sphere? Many thanks.

[riccardomarin]

Yes, you can also find the official repository with further details here.

Good question; I think that, as always happen with neural networks, it depends on your dataset. If you have enough and good data, generalization is possible. However, if you use low-pass spectrum (i.e., the first 30 eigenvalues), it is also true that they are good to encode global deformation and not details. Hence, it could be difficult recovering facial features that characterize heads from spheres (or at least achieving sharp results, but this also depends on the application you have in mind). You may find interesting this other work (not yet published), where, to overcome this limit, we rely on localized spectra.

Let me know if you have any other questions!

[YanhaoZhang]

Thanks a lot, @riccardomarin. That is really helpful for my research. I will try the robust_laplacian and read the paper. Really appreciate your help!

[YanhaoZhang]

Hi @riccardomarin I did some tests on the robust_laplacian you recommended. It is really useful! Thanks a lot. For the differentiation of mesh spectrum computation we discussed before, I am wondering if you could please help ask your friend for some suggestions? I realise that I may still need this. I am doing something similar as your paper. I would like to deform a mesh (new vertices) using the mesh spectrum as one of the constraints. Even if the computation is slow, it would be still very helpful for me as long as the spectrum calculation can be backpropagated. Thanks a lot.

[lucmos]

Hello @YanhaoZhang, You can find a differentiable implementation of the Laplacian Operator with Linear FEM here.

The notebook contains the PyTorch and Tensorflow implementation, some benchmarks, and some notes. Please keep in mind that it was written ~2 years ago, thus probably some parts could be improved :)

[YanhaoZhang]

Hi @lucmos, thanks a lot for your prompt reply. I will try spectraldifferentiation. This is really helpful for me! :smile: