zero gradient

[mks0601]

Can this code be used for training? I printed gradients of weights of the last layer, and they are always zero.

[gsygsy96]

I never met this problem...

[ShichenLiu]

If you set aggr_funcs or dist_func to "hard", then it's normal that there is no gradient, since standard rendering does not have gradient. Otherwise it's kind of weird.

[mks0601]

I think it is because of very sharp sigmoid function because of sigma value.

[mks0601]

Did you check this code can be used in training stage? I successfully integrated this code to my project, however it encounters cuda illegal memory access when backwarding. When I use nerual 3D mesh renderer (https://github.com/daniilidis-group/neural_renderer) and rest of the code is exactly same, it is alright. Can you give me some help?

[ShichenLiu]

One possible reason I think of is the texture format. We use a different texture format than NMR, which is more memory efficient. Can you please make sure the texture is the same as in the demo?

[mks0601]

Thank you for your help. I've used vertex texture type and just removed RGB part in cuda code because only silhouette part is what I need. Problem resolved.

[gsygsy96]

You mean the right format is 'vetex'? In your demo, texture format is 'surface', and the format meaning seems same with NMR. Can you explain the difference of texture format more clear?

[mks0601]

No. I think using vertex as a texture type was the reason of error. It seems like vertex means RGB colors are defined for each vertex. On the other hand, surface texture is defined as a small patch.

[gsygsy96]

I use 'vertex' or 'surface' format, and view textures's gradient. If using 'vertex' format, nearly 3000 texture points has gradient, others remain zero. If using 'surface' format, only one texture part has gradient...

[gsygsy96]

To correct this, when using surface format, only some fixed texture parts have gradients (their index numbers remain unchanged during training).

[mks0601]

Oh I only checked grads of face_vertex part. I've not checked grads of texture parts.

[gsygsy96]

Vertex always have gradient. Did you check grads of silhouette parts in your experiment? @mks0601 BTW, I'm still confused on the texture difference between NMR and SoftRas... In my mind, they have same texture format. @mks0601 @ShichenLiu

[mks0601]

What I mean was silhouette part. It is fine.

[ShichenLiu]

Hi @mehameha998 , NMR using texture format as [num_faces, texture_size, texture_size, texture_size, 3], basically they project the texture space into a 3D cube. In our implementation, we setup a UV coordinate for each face, and project the texture into [num_faces, texture_size^2, 3]. I will push a pdf with detailed explanation later for better understanding.

[sunshineatnoon]

To correct this, when using surface format, only some fixed texture parts have gradients (their index numbers remain unchanged during training).

Hi, I met the same problem, have you found a solution? Seems only some faces got gradients.

[mks0601]

I solve this by setting minimum and maximum depth value (maybe near and far?) based on my case.