Wrong Pointcloud Normals estimation if Poincloud has Nans

[bidbest]

Describe the bug Estimating Normals of a Pointcloud with Nans, results in wrong Normals (and no worning). Function: _Pointcloud.estimatenormals()

To Reproduce To reproduce the error, please download the attached pointcloud. Extract the .zip file and load the pointcloud with numpy. To reproduce:

import open3d as o3d
import numpy as np

# Your path to the attached npy file
points_path = "/path/to/pointcloud.npy"

points = np.load(points_path)
pc = o3d.geometry.PointCloud()
pc.points = o3d.utility.Vector3dVector(points[:,:3]) # We don't need the alpha color

# Index of non Nan points. Given the sensor used, we can assume that a point is either [nan, nan, nan], or valid.
# No cases like [x, Nan, z]
ind = np.where(np.isnan(points[:,0])==False)[0]

########################################################################
# For this case, we remove Nans and compute normals
pc2 = pc.select_by_index(ind)
pc2.estimate_normals(o3d.geometry.KDTreeSearchParamHybrid(radius=0.01, max_nn=30))
pc2.orient_normals_towards_camera_location([0,0,0])
pc2 = pc2.voxel_down_sample(0.005) # Down-sample for visualization

# The normals are computed as expected
o3d.visualization.draw_geometries([pc2], point_show_normal=True)

########################################################################
# For this case, we don't remove the Nans and compute the normals
pc.estimate_normals(o3d.geometry.KDTreeSearchParamHybrid(radius=0.01, max_nn=30))
pc.orient_normals_towards_camera_location([0,0,0])
# We still need to remove Nans to be able to visualize the pointcloud
pc3 = pc.select_by_index(ind)
pc3.voxel_down_sample(0.005)  # Down-sample for visualization

# The normals are all orthogonal to the camera plane, and not the the points planes
o3d.visualization.draw_geometries([pc3], point_show_normal=True)

Expected behavior Quick: return a warning, mentioning estimate normals doesn't support Pointclouds with Nans. Ideal: Ignore Nans in the computation, compute normals for valid points, and assign Nans for Nan points

Screenshots

Environment (please complete the following information):

• Operating system: Ubuntu 18.04
• Python version: Python 3.6.9
• Open3D version: '0.13.0'
• Is this remote workstation?:No
• How did you install Open3D?:pip
• Compiler version (if built from source): Nan

Additional context Please use this Pointcloud to reproduce the error ordered_pointcloud.zip

[bidbest]

Also thanks for the great work :) Please let me know if I missed something, or if you need more context!

[martinakos]

I'm dealing with the same issue. Are there any workarounds for this? I'm creating a pointcloud from a depth image using the project_valid_depth_only = False parameter in PointCloud.create_from_depth_image. I do this because I want to be able to recreate the original size image with the agreements of the normals to a vector. I can only get correct normals when project_valid_depth_only = True but then I can recreate the original image size.

[martinakos]

I found a workaround to get the correct normals for pcd, a bit tedious though:

        pcd = o3d.geometry.PointCloud.create_from_depth_image(
            o3d.geometry.Image(depth_image), 
            self.intrinsic, 
            self.T_cw,
            self.depth_scale,
            project_valid_depth_only = False)   

        points = np.asarray(pcd.points).T
        indices_not_nans = np.nonzero(np.all(np.isnan(points) == False, axis=0))[0]
        pcdn = pcd.select_by_index(indices_not_nans)               
        pcdn.estimate_normals()        
        normals_pcdn = np.asarray(pcdn.normals).T

        pcd.estimate_normals()  
        normals_pcd = np.asarray(pcd.normals).T
        normals_pcd[:,indices_not_nans] = normals_pcdn 
        pcd.normals = o3d.utility.Vector3dVector(normals_pcd.T)