Open CanCanZeng opened 3 months ago
Hi, indeed 2DGS's densitification metric relies on the gradient of the 3D point but I believe that the absGrad can be implant into the 3D gradient as well.
Sorry, I don't understand what you mean. Do you mean that the gradient of viewspace_point_tensor
in 2DGS is actually the gradient of loss relative to the 3D point?
You may check the code for further adaption. https://github.com/hbb1/diff-surfel-rasterization/blob/main/cuda_rasterizer/backward.cu#L639-L640
I'm sorry that I missed this detail when I read the paper. Thank you for your patient reply.
Like AbsGS, you can add homodirectional gradient accumulation to the backward kernel. Then, screenspace_points (defined in gaussian_renderer) must be modified as follows.
screenspace_points = torch.zeros_like(pc.get_xyz, dtype=pc.get_xyz.dtype, requires_grad=True, device="cuda") + 0
-edited
screenspace_points = torch.zeros((pc.get_xyz.shape[0], 4), dtype=pc.get_xyz.dtype, requires_grad=True, device="cuda") + 0
Results: (top: org 2D GS, bottom: w/ homodirection gradient)
Exactly. I add some hightlights.
Hi @hwanhuh thank you for your advice. Your result looks very good, but I tried the modification you made modified and the result is worse than the original version. May I ask why this is? In fact, I have made almost the same modifications as you and found that the result is incorrect. The value of grads_abs
is smaller than the value of grads
, so I came to consult the author.
@hwanhuh I also tried to transplant the MCMC algorithm into the 2DGS algorithm, which can also solve the problem of blurry cluttered areas. However, the MCMC algorithm introduces a new problem, which is that in areas with the sky as the background, if the foreground is a structure full of details such as tree branches, the reconstruction quality of this part will become blurry. Do you have a solution to this problem?
@CanCanZeng I plan to publish the 2D GS application project of AbsGS / MCMC, including other implementation details, on my GitHub, but not immediately. I will let you know when the code is organized.
Early on in my experiments applying MCMC, I observed a similar trend. ( In my opinion) This is because, since the relocation alone does not provide an appropriate update to the low-density region, the noise term needs to be carefully designed for MCMC. Directly applying MCMC's noise design to 2D GS is improper for me.
It's really kind of you to share your findings @hwanhuh
Hello, I found that 2DGS may reconstruct some very cluttered areas poorly. If you observe the distribution pattern of the point cloud, the point cloud in these areas is very sparse. The original 3DGS also has this problem, so someone proposed the AbsGS algorithm to improve the densification strategy. I encountered difficulties when trying to transplant this improvement into 2DGS, because the rendering of 2DGS is different from 3DGS, and the gradient of the 2D projection point is 0 in most cases. Does the author have any good suggestions for this problem?
Note the lower right corner of the picture below