Extracting geometry from a NeRF

[KleinXin]

In the tensorflow implementation, it has a function that the geometry of the scene can be extracted as cubes.

https://github.com/bmild/nerf https://github.com/bmild/nerf/blob/master/extract_mesh.ipynb

Is it also possible in this pytorch project？

[LHXhh]

I have the same question. Have anyone succeeded?

[djx99]

Do you solve it?

[KleinXin]

Do you solve it?

Not yet

[djx99]

I extracted the mesh based on this file. You can directly load the pytorch version of the pre-trained model, and then perform calculations according to these codes in extract_mesh.ipynb `N = 256 t = np.linspace(-1.2, 1.2, N+1)

query_pts = np.stack(np.meshgrid(t, t, t), -1).astype(np.float32) print(query_pts.shape) sh = query_pts.shape flat = query_pts.reshape([-1,3])

def batchify(fn, chunk): if chunk is None: return fn def ret(inputs): return tf.concat([fn(inputs[i:i+chunk]) for i in range(0, inputs.shape[0], chunk)], 0) return ret

fn = lambda i0, i1 : net_fn(flat[i0:i1,None,:], viewdirs=np.zeros_like(flat[i0:i1]), network_fn=render_kwargs_test['network_fine']) chunk = 1024*64 raw = np.concatenate([fn(i, i+chunk).numpy() for i in range(0, flat.shape[0], chunk)], 0) raw = np.reshape(raw, list(sh[:-1]) + [-1]) sigma = np.maximum(raw[...,-1], 0.)

print(raw.shape) plt.hist(np.maximum(0,sigma.ravel()), log=True) plt.show()`

[LHXhh]

Thank you very much!

------------------ 原始邮件 ------------------ 发件人: "yenchenlin/nerf-pytorch" @.>; 发送时间: 2022年9月9日(星期五) 中午11:13 @.>; @.**@.>; 主题: Re: [yenchenlin/nerf-pytorch] Extracting geometry from a NeRF (Issue #68)

I extracted the mesh based on this file. You can directly load the pytorch version of the pre-trained model, and then perform calculations according to these codes in extract_mesh.ipynb `N = 256 t = np.linspace(-1.2, 1.2, N+1)

query_pts = np.stack(np.meshgrid(t, t, t), -1).astype(np.float32) print(query_pts.shape) sh = query_pts.shape flat = query_pts.reshape([-1,3])

def batchify(fn, chunk): if chunk is None: return fn def ret(inputs): return tf.concat([fn(inputs[i:i+chunk]) for i in range(0, inputs.shape[0], chunk)], 0) return ret

fn = lambda i0, i1 : net_fn(flat[i0:i1,None,:], viewdirs=np.zeros_like(flat[i0:i1]), network_fn=render_kwargs_test['network_fine']) chunk = 1024*64 raw = np.concatenate([fn(i, i+chunk).numpy() for i in range(0, flat.shape[0], chunk)], 0) raw = np.reshape(raw, list(sh[:-1]) + [-1]) sigma = np.maximum(raw[...,-1], 0.)

print(raw.shape) plt.hist(np.maximum(0,sigma.ravel()), log=True) plt.show()`

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

[Miaosheng1]

Just for Reference!

Extract Mesh

        NeRF = render_kwargs_test['network_fine']
        N, chunk = 256, 1024*64

        t = np.linspace(-1.2, 1.2, N + 1)
        query_points = np.stack(np.meshgrid(t, t, t), -1).astype(np.float32)
        print(query_points.shape)
        flat = torch.from_numpy(query_points.reshape([-1, 3])).to(device)

        query_fn = render_kwargs_test['network_query_fn']

        sigma = []
        for i in range(0, flat.shape[0], chunk):
            pts = flat[i:i+chunk,None,:]
            viwedirs = torch.zeros_like(flat[i:i+chunk])
            raw = query_fn(pts, viwedirs, NeRF)
            sigma.append(raw[...,-1])
        density = torch.concat(sigma,dim=0).detach().cpu().numpy().squeeze()
        plt.hist(np.maximum(0, density), log=True)
        plt.savefig('density.png')

        import mcubes
        import trimesh
        threshold = 50.

        vertices, triangles = mcubes.marching_cubes(density.reshape(257,257,-1), threshold)
        print('done', vertices.shape, triangles.shape)
        mesh = trimesh.Trimesh(vertices, triangles)
        mesh.export('000.ply')