Other options on top of SDF

[wangyida]

Seems that works presenting the surface in unsigned distance field (UDF) such as NeUDF* and NeUDF is more promising for presenting slender and thin structures. Do you any plan towards structuring a practical way to implement it there in NSR?

[bennyguo]

I'm not very familiar with the differentiable rendering approaches used in these UDF papers. Do they differ much from NeRF and NeuS rendering? Currently I'm short on time to do this, but will be appreciated if you could implement a new system for things like this.

[wangyida]

@bennyguo Have geeked around the published UDF works, seems to be quite suitable for single object surface extracting, but I haven't make it work for unbounded scenes with torch_lighting. But instead my implementation with HF-NeuS seems to be reasonable, is it one of the interest for your project? Basically I tried with projecting SDF to transmittance instead of weight as mentioned in the paper.

[bennyguo]

@wangyida Sounds great! Did you try other tricks/regularizations proposed in the HF-NeuS paper? or you just tried with the transmittance formulation?

[wangyida]

Hi @bennyguo I have only implemented the transmittance formulation here, following the edcription on page 5 in HF-NeuS.

# cdf = sigmoid(sdf * inv_s) as proposed in HF-NeuS
cdf = torch.sigmoid(torch.unsqueeze(sdf, -1) * inv_s)
# e = sigma * step
e = inv_s * (1 - cdf) * (-iter_cos) * self.render_step_size
# alpha-composition, where alpha = 1 − exp(−sigma * step)
alpha = (1 - torch.exp(-e)).view(-1).clip(0.0, 1.0)

Cross entropy loss on weights are added to make weights and opacity more confident. The high frequency network need addtional parametric model to get it described, so I didn't add it

[wangyida]

Tested with an NSR projection function as well, I assume that the projection function implemented by Benny is still NeuS instead of Instant-NSR. PSNR on Mipnerf360 is a bit higher.

# Instant-NSR applies 𝜋(.) upon the estimated sddfsvalues
prev_sdf_pi = torch.sigmoid(estimated_prev_sdf * inv_s) * (1 - torch.exp(-estimated_prev_sdf * inv_s))
next_sdf_pi = torch.sigmoid(estimated_next_sdf * inv_s) * (1 - torch.exp(-estimated_next_sdf * inv_s))
prev_cdf = torch.sigmoid(prev_sdf_pi * inv_s)
next_cdf = torch.sigmoid(next_sdf_pi * inv_s)
p = prev_cdf - next_cdf
c = prev_cdf
alpha = ((p + 1e-5) / (c + 1e-5)).view(-1).clip(0.0, 1.0)