hlwang1124 / SNE-RoadSeg

SNE-RoadSeg for Freespace Detection in PyTorch, ECCV 2020
https://sites.google.com/view/sne-roadseg
MIT License
312 stars 76 forks source link

Problem of producing normal from a depth image #28

Closed RobbyHuang closed 3 years ago

RobbyHuang commented 3 years ago

Tanks for your sharing!

I try to use the SNE module to produce a normal image from a depth, but It seems that the quality of the result is not good. The depth image I used is from the YCB_Video. 000001-depth

000001-normal

depth_image = cv2.imread(depth_path, cv2.IMREAD_ANYDEPTH)
sne_model = SNE() 
camParam = torch.tensor([[1.066778e+03, 0.000000e+00, 3.129869e+02],
                                           [0.000000e+00, 1.067487e+03, 2.413109e+02],
                                           [0.000000e+00, 0.000000e+00, 1.000000e+00]], dtype=torch.float32)  # camera parameters
normal = sne_model(torch.tensor(depth_image.astype(np.float32)/1000), camParam)
normal_image = normal.cpu().numpy()
normal_image = np.transpose(normal_image, [1, 2, 0])
cv2.imwrite(normal_path, cv2.cvtColor(255 * (1 + normal_image) / 2, cv2.COLOR_RGB2BGR))

Could you give me some advice?Thanks!

hlwang1124 commented 3 years ago

Hi, @RobbyHuang . I think this is due to the invalid pixels in the depth image. A possible solution is to refine the depth image.

RobbyHuang commented 3 years ago

Hi,@hlwang1124 Thanks for your reply. Have you tried to use the SNE module to convert the depth image produced by the camera to normal? I found the same problem, when I convert the depth image produced by intel Realsense D435i to normal. Have you tried to refine the depth image? Can you recommend some related algorithms? Thanks!

hlwang1124 commented 3 years ago

Yes, I tried using the SNE module to convert Intel RealSense depth images to the corresponding surface normal, as shown here. Although there also exist invalid pixels, I found that they have little influence on the drivable area and road anomaly detection. Therefore, I have not tried to refine the depth image.