search

facebookresearch / pytorch3d

PyTorch3D is FAIR's library of reusable components for deep learning with 3D data
https://pytorch3d.org/
Other
8.81k stars 1.32k forks source link

How to optimize point radiuses #1562

Closed Kiord closed 1 year ago

Kiord commented 1 year ago

Hello, thank you for this powerful library !

I am trying to optimize a point cloud positions AND radiuses with differentiable rendering (with a PointsRenderer). The positions are correctly optimized but the radiuses remain the same at each iteration, even though they have strong impact on the rendered images.

Does the PointsRenderer allow for per point radius backpropagation ? If so, how ?

Kiord commented 1 year ago

It appears that the point positions is the only data for which the gradient is computed : https://github.com/facebookresearch/pytorch3d/blob/f68371d3988cab39e1730281f97e91b62887b2a6/pytorch3d/renderer/points/rasterize_points.py#L217-L240

However with the Pulsar backend, the gradient of the radiuses and other fields seems to be computed : https://github.com/facebookresearch/pytorch3d/blob/f68371d3988cab39e1730281f97e91b62887b2a6/pytorch3d/renderer/points/pulsar/renderer.py#L170-L273

I will try to do my stuff with PulsarPointsRenderer instead of PointsRenderer. Closing this issue.