Only Point Clouds in Grayscale after recording with azure_kinect_recorder.py

[Me817]

Hello,

Describe the bug I recorded some data with the azure_kinect_recorder.py and tried to visualize them in the Open3D Visualizer. Unfortunately all my point clouds are visualized without color just in grayscale. I tried every combination of color resolution and depht mode. But evertime the same problem occurs. The depth or the relations in height and width of the object shown in the point cloud aren't represeted correctly. The distance between the object is too small in the point clouds.

To Reproduce

1. Connect a Azure Kinect camera to the PC.
2. Run azure_kinect_recorder.py and record a scene.

3. Split the mkv file into the frames with azure_kinect_mkv_reader.py.

   while not self.mkv_reader.is_eof() and not self.stopped:
               if self.play:
                   # noinspection SpellCheckingInspection
                   rgbd = self.mkv_reader.next_frame()
                   if rgbd is None:
                       continue
   
                   if output_dir is not None:
                       color_img_filename = '{0}/color/{1:05d}.jpg'.format(output_dir, index)
                       print('Writing to {}'.format(color_img_filename))
                       o3d.io.write_image(color_img_filename, rgbd.color)
   
                       depth_img_filename = '{0}/depth/{1:05d}.png'.format(output_dir, index)
                       print("Writing to {}".format(depth_img_filename))
                       o3d.io.write_image(depth_img_filename, rgbd.depth)
   
                       index += 1

according to "examples/python/ReconstructionSystem/sensors/azure_kinect_recorder.py"

4. Generate point clouds (.pcd) out of the color and depth images:

   color_raw = o3d.io.read_image(self.color_dir + color)
           depth_raw = o3d.io.read_image(self.depth_dir + depth)
           # noinspection PyArgumentList
           rgbd_image = o3d.geometry.RGBDImage.create_from_color_and_depth(color=color_raw, depth=depth_raw)
           # noinspection PyArgumentList
           point_cloud = o3d.geometry.PointCloud.create_from_rgbd_image(image=rgbd_image,
                                                                        intrinsic=o3d.camera.PinholeCameraIntrinsic(
                                                                            o3d.camera.PinholeCameraIntrinsicParameters.PrimeSenseDefault),
                                                                        extrinsic=np.array(
                                                                            [[1., 0., 0., 0.], [0., 1., 0., 0.],
                                                                             [0., 0., 1., 0.], [0., 0., 0., 1.]]))
           point_cloud.transform([[1, 0, 0, 0], [0, -1, 0, 0], [0, 0, -1, 0], [0, 0, 0, 1]])
           point_cloud_filename = "{0}\\pcd\\{1}.pcd".format(pcd_dir, filename)
           o3d.io.write_point_cloud(point_cloud_filename, point_cloud)
           print("Writing to {}".format(point_cloud_filename))

5. Visualize the point clouds in the Open3D Visualizer (I use VisualizerWithCallback()):

   vis = o3d.visualization.VisualizerWithKeyCallback()
   vis.create_window("Visualizer", 960, 540)
   read_pcd = o3d.io.read_point_cloud(self.pcd_list[idx])
   vis.add_geometry(read_pcd)

Expected behavior I am wondering why the point clouds are in grayscale and not in the colors I recorded in the mkv and wrote into the color images. The point clouds in the documentation are colored and not grayscale. see: Documentation Point Cloud

Screenshots

Environment (please complete the following information):

• Operating system: Windows 10 64-bit
• Python version: Python3.7.4
• Open3D version: 0.10.0.1
• Is this remote workstation?: no
• How did you install Open3D?: pip

I do not know how to visualize my point clouds in color. Thank you in advance.

[Me817]

With the script provided by @kaixin-bai I can visualize the point cloud I capture by this script with the right colors. BUt I do not know how to transfer it on my code. Does anybody have an idea? Thank you in advance.

[kaixin-bai]

http://www.open3d.org/docs/latest/python_api/open3d.geometry.RGBDImage.html?highlight=create_from_color_and_depth#open3d.geometry.RGBDImage.create_from_color_and_depth

static create_from_color_and_depth(color, depth, depth_scale=1000.0, depth_trunc=3.0, convert_rgb_to_intensity=True)

to keep the rgb data you need to set "convert_rgb_to_intensity" to False.

[Me817]

@kaixin-bai Thank you for this advice. It helped a lot. I tried to set the depth_scale to 100.0 but in this case I can not save the point clouds. I always get:

[Open3D WARNING] Write PCD failed: unable to generate header.
Writing to ..\..\..\Desktop\record4\rgb_pcd5\00000.pcd

Is it not possible to set the scale lower than 1000.0? Thank you in advance.

[kaixin-bai]

@Me817 maybe more details? from which function did you get the warning?

[Me817]

I have no more details. That is the message I receive. I get no information about the function which causes this warning. I can give you the part of my code where I try to save the point clouds:

rgbd_image = o3d.geometry.RGBDImage.create_from_color_and_depth(color=color_raw, depth=depth_raw, depth_scale=100.0, depth_trunc=3.0, convert_rgb_to_intensity=False)
intrinsic = o3d.camera.PinholeCameraIntrinsic(1280, 720, 601.1693115234375, 600.85931396484375, 637.83624267578125, 363.8018798828125)
point_cloud = o3d.geometry.PointCloud.create_from_rgbd_image(rgbd_image, intrinsic)
point_cloud.transform([[1, 0, 0, 0], [0, -1, 0, 0], [0, 0, -1, 0], [0, 0, 0, 1]])
point_cloud_filename = "{0}\\rgb_pcd5\\{1}.pcd".format(pcd_dir, filename)
o3d.io.write_point_cloud(point_cloud_filename, point_cloud)

The message I posted above is repeated for every point cloud I try to save. And I only receive the warning if I set depth_scale to 100.0 for 1000.0 or 10000.0 I do not. I hope these information help. Thank you in advance.

[kaixin-bai]

I have no more details. That is the message I receive. I get no information about the function which causes this warning. I can give you the part of my code where I try to save the point clouds:

rgbd_image = o3d.geometry.RGBDImage.create_from_color_and_depth(color=color_raw, depth=depth_raw, depth_scale=100.0, depth_trunc=3.0, convert_rgb_to_intensity=False)
intrinsic = o3d.camera.PinholeCameraIntrinsic(1280, 720, 601.1693115234375, 600.85931396484375, 637.83624267578125, 363.8018798828125)
point_cloud = o3d.geometry.PointCloud.create_from_rgbd_image(rgbd_image, intrinsic)
point_cloud.transform([[1, 0, 0, 0], [0, -1, 0, 0], [0, 0, -1, 0], [0, 0, 0, 1]])
point_cloud_filename = "{0}\\rgb_pcd5\\{1}.pcd".format(pcd_dir, filename)
o3d.io.write_point_cloud(point_cloud_filename, point_cloud)

The message I posted above is repeated for every point cloud I try to save. And I only receive the warning if I set depth_scale to 100.0 for 1000.0 or 10000.0 I do not. I hope these information help. Thank you in advance.

maybe you can also upload the files you are using, so that the other guys can help you.

[theNded]

Please keep depth_scale unchanged. There is no reason to change it randomly.