Question about modeling a 3d shape using marching cube

[vincenthesiyuan]

Hi, Thanks for your great work. I trained an OctreeSDF model on LOD5, and want to do marching cube that similar to SIREN. Unfortunately, it dosen't work. The output .ply model have no shape and all of noise in the space.

Sorry for the broken english and my stupid question. Looking forward to your answer.

[joeylitalien]

Hi @vincenthesiyuan,

You can try adding the following app in sdf-net/app. This is untested with the current branch but should work, as long as you call it with the same options you used to train.

import sys
import os

sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))

import numpy as np
import torch
import torch.nn as nn

import trimesh
import mcubes

from lib.models import *
from lib.options import parse_options

def extract_mesh(args):
    # Prepare directory
    ins_dir = os.path.join(args.mesh_dir, name)
    if not os.path.exists(ins_dir):
        os.makedirs(ins_dir)

    # Get SDFs
    with torch.no_grad():
        xx = torch.linspace(-1, 1, args.mc_resolution, device=device)
        pts = torch.stack(torch.meshgrid(xx, xx, xx), dim=-1).reshape(-1,3)
        chunks = torch.split(pts, args.batch_size)
        dists = []
        for chunk_pts in chunks:
            dists.append(net(chunk_pts).detach())

    # Convert to occupancy
    dists = torch.cat(dists, dim=0)
    grid = dists.reshape(args.mc_resolution, args.mc_resolution, args.mc_resolution)
    occupancy = torch.where(grid <= 0, 1, 0)

    # Meshify
    print('Fraction occupied: {:.5f}'.format((occupancy == 1).float().mean().item()))
    # vertices, triangles = mcubes.marching_cubes(occupancy.cpu().numpy(), 0.5) # Original post, small bug
    vertices, triangles = mcubes.marching_cubes(occupancy.cpu().numpy(), 0)

    # Resize + recenter
    b_min_np = np.array([-1., -1., -1.])
    b_max_np = np.array([ 1.,  1.,  1.])
    vertices = vertices / (args.mc_resolution - 1.0) * (b_max_np - b_min_np) + b_min_np

    # Save mesh
    mesh = trimesh.Trimesh(vertices, triangles)
    mesh_fname = os.path.join(ins_dir, f'mc_res{args.mc_resolution}.obj')
    print(f'Saving mesh to {mesh_fname}')
    mesh.export(mesh_fname)

if __name__ == '__main__':
    # Parse
    parser = parse_options(return_parser=True)
    app_group = parser.add_argument_group('app')
    app_group.add_argument('--mesh-dir', type=str, default='_results/render_app/meshes',
                           help='Directory to save the mesh')
    app_group.add_argument('--mc-resolution', type=int, default=256,
                           help='Marching cube grid resolution.')
    args = parser.parse_args()

    # Pick device
    use_cuda = torch.cuda.is_available()
    device = torch.device('cuda' if use_cuda else 'cpu')

    # Get model
    if args.pretrained is not None:
        name = args.pretrained.split('/')[-1].split('.')[0]
    else:
        raise ValueError('No network weights specified!')
    net = globals()[args.net](args)
    net.load_state_dict(torch.load(args.pretrained), strict=False)
    net.to(device)
    net.eval()

    # Run Marching Cubes
    extract_mesh(args)

[zhaoyuanyuan2011]

Hi @joeylitalien ,

Thank you for providing the exporter! I used the script and export a mesh, which has layered effect (like a voxel instead of smooth mesh) and is probably because the resolution of marching cube is too low.

However, I've used 256 and even 512 so res shouldn't be the cause. I noticed that the fraction occupied is only 0.04. Is it fraction too low? Is it possible that there are too many empty points (points that are not inside the mesh) so even a 512 res doesn't help smooth the mesh? As a comparison, the img rendered from sdf using sdf_renderer.py looks pretty smooth.

Thank you!

[YuanxunLu]

I met the same problem exactly using the code above. I found the keypoint here is that it uses occupancy values as marching cubes input instead of SDF values, which causes the striped mesh.

Therefore, changing the codes of marching cubes works. Specifically, replace the original code

vertices, triangles = mcubes.marching_cubes(occupancy.cpu().numpy(), 0.5)

with

vertices, triangles = mcubes.marching_cubes(grid.cpu().numpy(), 0)

The results of 256 resolution will be as follows:

Hope the above helps!

[zhaoyuanyuan2011]

Hi @YuanxunLu, thank you so much for the reply! I haven't noticed this 😂

[joeylitalien]

Welp, that teaches me to put untested code snippets! Thanks for the quick fix @YuanxunLu.