how to visualize outputs

[donghyunyoo]

Hi I want to visualize the 3D outputs in output_shape and output_color folders. How can I visualize them?

[syb7573330]

Hi, Thanks for reaching out! For visualization, I simply saved the 3D coordinates and color of occupied surface voxels, and used Unity to render them as small cubes centered at their corresponding 3D coordinates.

[rohaantahir]

@syb7573330 can you please elaborate on how to visualize the models that are generated by inference_shape.py. I am successfully able to get .h5 inferred files. kindly guide me thanks

[jackieyung]

Hi, Thanks for reaching out! For visualization, I simply saved the 3D coordinates and color of occupied surface voxels, and used Unity to render them as small cubes centered at their corresponding 3D coordinates.

Hi @syb7573330 Thanks for your suggestion. But I'm not familiar with Unity, could you provide a sample code for showing how to render the output results?

[rohaantahir]

@syb7573330 kindly tell us that how did you saved the 3D coordinates?. Thanks

[jackieyung]

@syb7573330 kindly tell us that how did you saved the 3D coordinates?. Thanks

Hi @rohaantahir Do you know how to render the output results? If you know, could you share me a sample code? Thanks a lot.

[jackieyung]

I plotted the 3D model by using ipyvolume.
The result you can see below:

The code is very simple, wish it can help you all:

import ipyvolume as ipv
import h5py
import numpy as np

file_name = 'data1/models/model_normalized_64.h5'
model = h5py.File(file_name, 'r')
data = np.array(model['data'][:])
X = data[:,:,:,0]
Y = data[:,:,:,1]
Z = data[:,:,:,2]

fig = ipv.figure()
ipv.volshow(X, controls=False, max_shape=64)
ipv.volshow(Y, controls=False, max_shape=64)
ipv.volshow(Z, controls=False, max_shape=64)
ipv.show()

[rohaantahir]

Hi, can you please elaborate on how to render the output in unity3D?

[uree]

Another way to visualize the models in python.

@jackieyung's answer works, but for me it lacked more control over rendering as well as the ability to export and save the shape in any of the popular formats for 3d modelling.

The example below is for shape inference only.

import numpy as np
import open3d as o3d
import h5py
import sys

file_name = 'model_normalized_64.h5'
model = h5py.File(file_name, 'r')

data = np.array(model['data'][:])

# you only really need one of the 3 3D arrays to get a basic shape
X = data[:,:,:,0]

# this gets you all the indexes of the elements in the array which are not -1 (empty voxels), which is what you need for open3d. you're not working with one value for each voxel in the 64x64x64 space anymore but only the coordinates of full voxels.
coordinates = np.argwhere(X != -1)

pcd = o3d.geometry.PointCloud()
pcd.points = o3d.utility.Vector3dVector(coordinates)

# save pointcloud to file 
o3d.io.write_point_cloud("pointcloud.ply", pcd)

# optimization https://towardsdatascience.com/5-step-guide-to-generate-3d-meshes-from-point-clouds-with-python-36bad397d8ba
pcd.estimate_normals()
distances = pcd.compute_nearest_neighbor_distance()
avg_dist = np.mean(distances)
radius = 3 * avg_dist
bpa_mesh = o3d.geometry.TriangleMesh.create_from_point_cloud_ball_pivoting(pcd,o3d.utility.DoubleVector([radius, radius * 2]))

# .obj output
o3d.io.write_triangle_mesh("object.obj", bpa_mesh )

[hui1123]

I use jupyter notebook, but I don't see the visual output? Besides, I received a warning“RuntimeWarning: invalid value encountered in true_divide gradient = gradient / np.sqrt(gradient[0]2 + gradient[1]2 + gradient[2]**2)” Can you do me a favor? Thanks @jackieyung