How do I take the derivative of vertexPos, when I take the derivative of vertexPos and I find that loss doesn't change

[mhaberma]

Did you plot the gradient values? Are they close to zero?

Our layer computes these gradients automatically. So there is no manual action required.

[Luckyalum]

Yes, the gradient values is zero. What could be the reason? Does it matter if I set albedoMode_attr to a foregroundMask？ Thanks.

[mhaberma]

Hi again,

yes the foregroundMask flag is the issue. For that branch we have not implemented the gradients.

I will add a warning in the code to let people know about that issue.

The workaround for know could be that you set all vertex colors to 1.0 and choose the vertexColor flag. This should work.

[Luckyalum]

Thank you for your reply. According to your method, there is a gradient but very small(only 0.02), so the descending speed is very slow. I am reproducing your paper (DeepCap), which mentioned using this tool to calculate Silhouette Loss. Can you describe how this tool is used in DeepCap?

[mhaberma]

Hi,

For DeepCap we used the rasterizer only to compute the depth maps and therefore the boundary vertices (which lie at the depth map discontinuity).

The silhouette loss itself was implemented separately using the distance transform representation as described in the paper (also checkout LiveCap for more details about the distance transform loss).

Best, -- Marc Habermann, M.Sc. Doctoral Researcher Max Planck Institute for Informatics Saarland Informatics Campus Campus E1 4 66123 Saarbruecken Germany http://people.mpi-inf.mpg.de/~mhaberma/

Am 17.03.2021 um 03:16 schrieb baoxzh @.***>:

Thank you for your reply. According to your method, there is a gradient but very small(only 0.02), so the descending speed is very slow. I am reproducing your paper (DeepCap), which mentioned using this tool to calculate Silhouette Loss. Can you describe how this tool is used in DeepCap?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/aytewari/GVV-Differentiable-CUDA-Renderer/issues/2#issuecomment-800740339, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANS5XCWNNPSGEDJLPQINEATTEAGI7ANCNFSM4W3J7C3Q.

[Luckyalum]

Hi, I did not see the method of rendering the depth map in this tool. Is there an API to render the depth map? Thank you.

[mhaberma]

We removed that one for the official release due to efficiency reasons. But instead of depth you can just query the model mask and check for a discontinuity there. This has the same effect.

Best, -- Marc Habermann, M.Sc. Doctoral Researcher Max Planck Institute for Informatics Saarland Informatics Campus Campus E1 4 66123 Saarbruecken Germany http://people.mpi-inf.mpg.de/~mhaberma/

Am 19.03.2021 um 08:20 schrieb baoxzh @.***>:

Hi, I did not see the method of rendering the depth map in this tool. Is there an API to render the depth map? Thank you.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/aytewari/GVV-Differentiable-CUDA-Renderer/issues/2#issuecomment-802612873, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANS5XCV47TXINPWCUBSDGI3TEL3NLANCNFSM4W3J7C3Q.

[Luckyalum]

Thank you for your reply. I found that the model mask has no gradient: ValueError: No gradients provided for any variable:['Varible:0'].

Has the gradient been implemented on the model mask?

[mhaberma]

In DeepCap, we fix the boundary vertices during one gradient step. So this part is non-differentiable.

It is similar to the visibility function in some differentiable renderers which is also assumed to not change during one gradient step.

Best, -- Marc Habermann, M.Sc. Doctoral Researcher Max Planck Institute for Informatics Saarland Informatics Campus Campus E1 4 66123 Saarbruecken Germany http://people.mpi-inf.mpg.de/~mhaberma/

Am 22.03.2021 um 04:16 schrieb baoxzh @.***>:

Thank you for your reply. I found that the model mask has no gradient: ValueError: No gradients provided for any variable:['Varible:0'].

Has the gradient been implemented on the model mask?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/aytewari/GVV-Differentiable-CUDA-Renderer/issues/2#issuecomment-803732793, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANS5XCVOKHI7VOELQEMIJW3TE2ZBZANCNFSM4W3J7C3Q.

[Luckyalum]

Hi, I queried the model mask and check for a discontinuity there. But how do I get the boundary vertices from discontinuity? I did not find the mapping between the boundary vertices and the depth map discontinuity

[MarcHabermann]

Hi,

You have to project the vertices onto the mask image.

Then check whether this vertex is visible or not.

If it is visible you do a local readout of the image location where the vertex projects onto and check if the image location has a gradient (a discontinuity is present).

If this is the case it is a boundary vertex.

Best,

-- Marc Habermann, M.Sc. Doctoral Researcher Max Planck Institute for Informatics Saarland Informatics Campus Campus E1 4 66123 Saarbruecken Germany http://people.mpi-inf.mpg.de/~mhaberma/

Am 27.03.2021 um 08:58 schrieb baoxzh @.***>:

 Hi, I queried the model mask and check for a discontinuity there. But how do I get the boundary vertices from discontinuity? I did not find the mapping between the boundary vertices and the depth map discontinuity

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.