Question regarding viewer_point_cloud.py

[kfarivar]

Hi I was looking at the viewer_point_cloud.py script is there a reason you use :

while True:
    capture = k4a.get_capture()
    if np.any(capture.depth) and np.any(capture.color):
        break

instead of just : capture = k4a.get_capture() It seems you want to make sure the depth and color cameras are both synchronized but I think I read they are synchronized via the hardware ?

also is there a reason for having two while loops ?

thank you

[lpasselin]

The first loop is probably to clear the first capture, which iirc is sometimes black for capture.color

If an image is badly processed by the SDK or by the C pyk4a code, it is possible capture.depth or capture.color will be None it is why this line is there.

The line is not required if the device configuration is synchronized. In the script, the configuration it is shown here: https://github.com/etiennedub/pyk4a/blob/master/example/viewer_point_cloud.py#L15

You must understand synchronized images are software synchronized. If there are dropped images (depth or color), the capture is not sent when configured as synchronized. More info here: https://microsoft.github.io/Azure-Kinect-Sensor-SDK/master/structk4a__device__configuration__t_a8208974f05d89fc1362c6a0900bdef4d.html#a8208974f05d89fc1362c6a0900bdef4d

[kfarivar]

Ok thanks it makes more sense.

Regarding the same script I also wanted to know if there is a way of mapping 2D masks/selection (on the image pixels) into the pointcloud. (I segment the objects in 2D and want to map that to the pointcloud) is that possible ? technically something like the capture.transformed_color method ? or somehow making the color_image_to_depth_camera method public ?

[lpasselin]

I think it is already available with from pyk4a import color_image_to_depth_camera?

[kfarivar]

I think I have to send the calibration to color_image_to_depth_camera ? and it is a private variable of capture I think.

[lpasselin]

I noticed a problem with transformed_depth_point_cloud (has color camera reference). Fixed here: https://github.com/etiennedub/pyk4a/pull/73 After this PR is merged, you will be able to modify viewer_point_cloud.py to replace capture.depth_point_cloud with capture.transformed_depth_point_cloud and capture.transformed_color with capture.color. Or replace capture.color with your segmentation/mask directly!

[kfarivar]

Ok so after the pull request I can just set my mask as capture.color and then call capture.transformed_color ?

and estimation when the PR will get merged ?

[lpasselin]

No.

In the point cloud example, directly replace the following:

Replace depth_point_cloud with transformed_depth_point_cloud

Replace color with segmentation mask.

On Thu., Oct. 15, 2020, 03:56 KiyarashFarivar, notifications@github.com wrote:

Ok so after the pull request I can just set my mask as capture.color and then call capture.transformed_color ?

and estimation when the PR will get merged ?

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/etiennedub/pyk4a/issues/71#issuecomment-708971591, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABTB77CWYBF2ZFSQPTHJ3J3SK2TJBANCNFSM4SQW5NOQ .