Closed maorp closed 2 years ago
Yes, unfortunately, the Cuda kernels for those transform operations only provide gradients to the positions currently, not to the matrix (that was all we needed for our use case). We should probably add that limitation to the docs, thanks for pointing that out!
The implementation of those kernels is here:
https://github.com/NVlabs/nvdiffmodeling/blob/main/src/renderutils/c_src/mesh.cu
So for now, if you need gradients to the matrix, please write the transform op in PyTorch (or use the use_python=True
flag as mentioned above). If should be straightforward to add gradients to the matrix as well, if this op is a performance bottleneck, but we haven't got around to it.
hi, when using renderutils.xfm_points (e.g. in render.py v_pos_clip = ru.xfm_points(mesh.v_pos[None, ...], mtx_in) - and trying to optimize camera pose in mtx_in - gradients does not flow back. Only when using use_python=True it worked.