luxonis / depthai-core

DepthAI C++ Library
MIT License
231 stars 126 forks source link

[Feature-Request] Depth Post-Processing: new "Mininum" decimation mode #808

Open matlabbe opened 1 year ago

matlabbe commented 1 year ago

Start with the why:

We are using the depth image from OAK-D camera to detect obstacles. However, for computation efficiency of the obstacle detection approach (costmap_2d's voxel layer), we don't need the whole 640x480 resolution of the OAK-D. We then configure the DecimationFilter to decimate by 4 the images. With the default decimation mode PIXEL_SKIPPING, the robot had difficulty detecting far thin obstacles that can actually be seen in the full resolution depth image, but somewhat ignored in output decimated depth image if the obstacles fall in the 3 pixels removed from decimation.

We tried other decimation modes:

We then created on host computer a simple algorithm to decimate the full resolution image using our new MINIMUM decimation mode. For example, when decimating by 4, it will iterate over each 4x4 ROI on depth image to find the closest non-zero pixel, then output that depth in output decimated image. This approach then ensures that the decimated image will always keep the closest obstacles.

Move to the what:

As described above, we would be nice that this MINIMUM decimation mode be implemented on the camera, to save CPU time on host computer.

Move to the how:

See above.

chengguizi commented 1 year ago

This is a very helpful suggestion!