Open ronly2460 opened 11 months ago
While this issue #66 mentioned that BARF struggles to reconstruct in NDC, I could not train even NeRF using NDC.
This is the function to convert rays to ndc rays in camera.py
def convert_NDC(opt,center,ray,intr,near=1): # shift camera center (ray origins) to near plane (z=1) # (unlike conventional NDC, we assume the cameras are facing towards the +z direction) center = center+(near-center[...,2:])/ray[...,2:]*ray # projection cx,cy,cz = center.unbind(dim=-1) # [B,HW] rx,ry,rz = ray.unbind(dim=-1) # [B,HW] scale_x = intr[:,0,0]/intr[:,0,2] # [B] scale_y = intr[:,1,1]/intr[:,1,2] # [B] cnx = scale_x[:,None]*(cx/cz) cny = scale_y[:,None]*(cy/cz) cnz = 1-2*near/cz rnx = scale_x[:,None]*(rx/rz-cx/cz) rny = scale_y[:,None]*(ry/rz-cy/cz) rnz = 2*near/cz center_ndc = torch.stack([cnx,cny,cnz],dim=-1) # [B,HW,3] ray_ndc = torch.stack([rnx,rny,rnz],dim=-1) # [B,HW,3] return center_ndc,ray_ndc
Additionally, the function below is generating ndc rays in vanilla-nerf.
def ndc_rays(H, W, focal, near, rays_o, rays_d): # Shift ray origins to near plane t = -(near + rays_o[...,2]) / rays_d[...,2] rays_o = rays_o + t[...,None] * rays_d # Projection o0 = -1./(W/(2.*focal)) * rays_o[...,0] / rays_o[...,2] o1 = -1./(H/(2.*focal)) * rays_o[...,1] / rays_o[...,2] o2 = 1. + 2. * near / rays_o[...,2] d0 = -1./(W/(2.*focal)) * (rays_d[...,0]/rays_d[...,2] - rays_o[...,0]/rays_o[...,2]) d1 = -1./(H/(2.*focal)) * (rays_d[...,1]/rays_d[...,2] - rays_o[...,1]/rays_o[...,2]) d2 = -2. * near / rays_o[...,2] rays_o = torch.stack([o0,o1,o2], -1) rays_d = torch.stack([d0,d1,d2], -1) return rays_o, rays_d
When I compare two methods, 2 is not multiplied in scale_x and scale_y.
2
scale_x
scale_y
Could you tell me the reason?
While this issue #66 mentioned that BARF struggles to reconstruct in NDC, I could not train even NeRF using NDC.
This is the function to convert rays to ndc rays in camera.py
Additionally, the function below is generating ndc rays in vanilla-nerf.
When I compare two methods,
2
is not multiplied inscale_x
andscale_y
.Could you tell me the reason?