Closed sunshineatnoon closed 2 years ago
Hi, I think it should be possible, I haven't tried it though.
The whole scene is bounded in the unit cube, scene is normalized such that the face is at an average of z=0.5 from the camera.
A different way to get a mesh that should be ok for faces, without having to samples a grid and do MC, as faces are very simple surfaces, is to render an expected depth map (use more ray samples though), unproject it to a point cloud and run poison reconstruction.
Hi, I think it should be possible, I haven't tried it though.
The whole scene is bounded in the unit cube, scene is normalized such that the face is at an average of z=0.5 from the camera.
A different way to get a mesh that should be ok for faces, without having to samples a grid and do MC, as faces are very simple surfaces, is to render an expected depth map (use more ray samples though), unproject it to a point cloud and run poison reconstruction.
Thanks so much for your response. Could you point me to which code you use to extract the mesh? Thanks!
The meshification in this image is done in Meshlab, screened Poisson surface reconstruction.
To backproject a depth image from our code, you can use the following code within the evaluation script: Here, depth should be a tensor of size H,W,1
def unproject_torch(depth, focal, world_coords=True,pose=None, name=None, H=512,W=512,save=True):
raster = meshgrid_xy(torch.arange(W), torch.arange(H))
u = raster[0]
v = raster[1]
x = (u - focal[2]) * depth.cpu() / ((focal[0]))
y = (v - focal[3]) * depth.cpu() / ((focal[1]))
z = depth.cpu()
pts = torch.dstack((x, y, z)).reshape(-1,3)
dims=3
if world_coords and pose is not None:
points_hom = torch.cat((pts, torch.ones(pts.shape[0], 1)), dim=1)#.to(depth.device)
pts = torch.matmul(torch.inverse(pose).cpu(), points_hom.transpose(1, 0)).transpose(0, 1)
dims=4
save_pc(pts,name,dims=dims)
return pts
def save_pc(tensor,name=None,dims=4):
from pathlib import Path
if name is None:
from datetime import datetime
now = datetime.now() # current date and time
date_time = now.strftime("%m_%d_%Y_%H:%M:%S")
fname = name if name is not None else date_time + ".obj"
print("saving point cloud %s" % fname)
Path(fname).write_text("\n".join([f"v {p[0]} {p[1]} {p[2]}" for p in tensor.reshape(-1,dims)]))
return
To generate a depth map, you'd have to use the rendering weights to get a weighted sum of the z values of the ray points. in volume_render_radiance_field, use
depth_map = weights * depth_values
depth_map = depth_map.sum(dim=-1)
That's great! Thanks so much for your help!
Hi, I was wondering if we can extract meshes from the learned NeRFs as in the original NeRF repo. Specifically, how to sample a valid cube? What's the range of the coordinates should be? Thanks!