JUGGHM / PENet_ICRA2021

ICRA 2021 "Towards Precise and Efficient Image Guided Depth Completion"
MIT License
326 stars 46 forks source link

lower lidar scanline input #70

Open hjl66662 opened 1 year ago

hjl66662 commented 1 year ago

I used the original 64 line depth map to generate a 16 line depth map and input it into the E network, resulting in a significant decrease in accuracy, with RMSE reaching nearly 4000,May I ask if there is a problem with the code I modified, or if the low lidar scanline input has caused such a significant decrease in accuracy

The following is the code I used to generate low lidar scanline depth maps

import numpy as np import cv2

def sample_lidar_lines( depth_map: np.ndarray, intrinsics: np.ndarray, keep_ratio: float = 1.0 ) -> np.ndarray: """ Takes in input a depth map generated by a 64 line lidar and sparsify the number of lines used, returning a sparse depth map with less lidar lines. Parameters

depth_map: array like
    sparse depth map of shape H x W x 1
intrinsics: array like
    the intrinsic parameters of shape 3 x 3
keep_ratio: float, default 1.0
    the sparsification parameter, 1.0 is 64 lines, 0.50 roughly 32 lines and so on.
Returns
-------
sparse_depth_map: array like
    the sparsified depth map of shape H x W x 1
"""

v, u, _ = np.nonzero(depth_map)
z = depth_map[v, u, 0]
points = np.linalg.inv(intrinsics) @ (np.vstack([u, v, np.ones_like(u)]) * z)
points = points.transpose([1, 0])

scan_y = points[:, 1]
distance = np.linalg.norm(points, 2, axis=1)
pitch = np.arcsin(scan_y / distance)
num_points = np.shape(pitch)[0]
pitch = np.reshape(pitch, (num_points, 1))

max_pitch = np.max(pitch)
min_pitch = np.min(pitch)
angle_interval = (max_pitch - min_pitch) / 64.0
angle_label = np.round((pitch - min_pitch) / angle_interval)
sampling_mask = angle_label % (1.0 / keep_ratio) == 0

final_mask = np.zeros_like(depth_map, dtype=bool)
final_mask[depth_map[..., 0] > 0] = sampling_mask
sampled_depth = np.zeros_like(final_mask, dtype=np.float32)
sampled_depth[final_mask] = depth_map[final_mask]
return sampled_depth