isl-org / Open3D

Open3D: A Modern Library for 3D Data Processing
http://www.open3d.org
Other
11.23k stars 2.28k forks source link

Wrong point cloud color when using RGBA images #5788

Open domef opened 1 year ago

domef commented 1 year ago

Checklist

Describe the issue

The point cloud has wrong colors when using source RGBA images depending on the alpha value. I don't know if the problem is linked to the point cloud creation or just to the visualization. This problem doesn't happen if the source image has just 3 channels.

This is the point cloud if using the RGB image: image This is the point cloud if using RGBA image with ALPHA_VALUE=0: image This is the point cloud if using RGBA image with ALPHA_VALUE=100: image This is the point cloud if using RGBA image with ALPHA_VALUE=255: image

Steps to reproduce the bug

import open3d as o3d
import numpy as np

np.random.seed(0)

ALPHA_VALUE = 0

rgb = (np.random.rand(512, 512, 3) * 255).astype(np.uint8)
alpha = np.full((512, 512, 1), fill_value=ALPHA_VALUE, dtype=np.uint8)
image = np.concatenate([rgb, alpha], axis=2)
depth = np.ones((512, 512), dtype=np.float32)
image_o3d = o3d.geometry.Image(image)
depth_o3d = o3d.geometry.Image(depth)
rgbd_image = o3d.geometry.RGBDImage.create_from_color_and_depth(
    image_o3d,
    depth_o3d,
    convert_rgb_to_intensity=False,
    depth_scale=1,
    depth_trunc=1000,
)
pc = o3d.geometry.PointCloud.create_from_rgbd_image(
    rgbd_image, o3d.camera.PinholeCameraIntrinsic()
)
o3d.visualization.draw_geometries([pc])

Error message

No response

Expected behavior

The point cloud should have the same color as the source image.

Open3D, Python and System information

- Operating system: Ubuntu 20.04
- Python version: Python 3.8.10
- Open3D version: 0.16.0
- System architecture: x86
- Is this a remote workstation?: no
- How did you install Open3D?: pip

Additional information

No response

ssheorey commented 1 year ago

alpha values are not supported by the visualization backend (filament), so we can't support them in Open3D. Please only provide an RGB image. This needs an error message....

Duplicate of this issue https://github.com/isl-org/Open3D/issues/1086

wheresjames commented 1 year ago

I think what's meant here is that the alpha channel is being written as the blue color component instead of being ignored, which would be more useful.

The issue is here.

    Eigen::Vector3d(pc[0], pc[(NC - 1) / 2], pc[NC - 1])

Using ((NC-1)/2)+(NC/3) might be a better choice for blue.