Issue with custom PointCloud on SVR

[pramishp]

When using the custom point cloud, I was getting a flat result as shown below.

Then I tested with the BMan0201-HD2-O04P05-S_3DSV_H250_W250_res256.npz file you have provided. The result was amazing.

To find out the issue, I read the BMan0201-HD2-O04P05-S_3DSV_H250_W250_res256.off file and used the vertices to compute the npz file using code below. The output was not flat as before but the reconstruction was different and parts above some point were missing as shown below.

So, the issue should be with the voxelization. How to do it correctly? Thank you very much for this awesome project.

bb_min = -0.5 
bb_max = 0.5
res = 256

def create_grid_points_from_bounds(minimun, maximum, res):
    x = np.linspace(minimun, maximum, res)
    X, Y, Z = np.meshgrid(x, x, x, indexing='ij')
    X = X.reshape((np.prod(X.shape),))
    Y = Y.reshape((np.prod(Y.shape),))
    Z = Z.reshape((np.prod(Z.shape),))

    points_list = np.column_stack((X, Y, Z))
    del X, Y, Z, x
    return points_list

def voxelized_pointcloud_sampling(path, grid_points, kdtree):
    off_path = path
    out_file = r"./" + off_path.split(os.sep)[-1]+".npz"

    point_cloud = trimesh.load(path)
    vertices = point_cloud.vertices

    occupancies = np.zeros(len(grid_points), dtype=np.int8)

    _, idx = kdtree.query(vertices)
    occupancies[idx] = 1

    compressed_occupancies = np.packbits(occupancies)

    np.savez(out_file, point_cloud=vertices, compressed_occupancies = compressed_occupancies, bb_min = bb_min, bb_max = bb_max, res = res)
    print(out_file)
    print('Finished {}'.format(path))

def convert_to_npz(path):

    grid_points = create_grid_points_from_bounds(bb_min, bb_max, res)
    kdtree = KDTree(grid_points)
    voxelized_pointcloud_sampling(path, grid_points, kdtree)

This is the test script I have used :

gen = Generator(net, 0.5, exp_name, checkpoint=checkpoint, resolution=256, batch_points=1000000)
occupancies = np.unpackbits(np.load(voxel_path)['compressed_occupancies']) # voxel_path = npz file path 
input = np.reshape(occupancies, (res,) * 3)
data = {}
data["inputs"] = np.array(input, dtype=np.float32)
logits = gen.generate_mesh(data)
mesh = gen.mesh_from_logits(logits)

[jchibane]

Hi @pramishp ,

please have a look at issue #27 - that might get you started. As mentioned in issue #23 an implementation into IF-Nets for direct usage of custom point clouds is desirable - but not there yet. A PR or an update from my side might come at a later stage.

Best, Julian