Inconsistency in GPU and CPU TSDF

[jsaisagar]

Hi,

When I am using CUDA TSDF using python, I am getting 0 vertices extracted. While the same dataset, TSDF on CPU is succesfull and am getting nice mesh. I am attaching a sample depth frame for one camera for reference. May I know what might cause this issue? Here are the parameters I am using for both CPU and GPU:

voxel_size = 4/1024 sdf_trunc = 0.04 block_resolution = 16 initial_block_count = 800 print('here 1') volume = o3d.t.geometry.TSDFVoxelGrid( { 'tsdf': o3d.core.Dtype.Float32, 'weight': o3d.core.Dtype.UInt16, 'color': o3d.core.Dtype.UInt16 }, voxel_size=voxel_size, sdf_trunc=sdf_trunc, block_resolution=block_resolution, block_count=initial_block_count, device=device) Integrate function: volume.integrate( depth,color, intrinsic_gpu, extrinsic_alignment_gpu,1000,5.0)

I have tried for different voxel sizes and intial block count as suggested in example. However, with any setting vertices are always 0.

PS: I captured depth map using Azure Kinect MKV reader Thanks for any help, Sai

[theNded]

Is there any error thrown from the GPU side?

[jsaisagar]

Thanks for the reply. I didn't get any gpu error

[theNded]

That is weird... Would you mind sharing a minimal dataset that can reproduce the error?

[jsaisagar]

Please find the data in link here.

Extrinsic matrix is identity.

[theNded]

This issue is caused by https://github.com/intel-isl/Open3D/blob/master/cpp/open3d/t/geometry/kernel/GeometryMacros.h#L371 and https://github.com/intel-isl/Open3D/blob/master/cpp/open3d/t/geometry/kernel/TSDFVoxelGridShared.h#L442, where we aggressively filter TSDF outliers by assuming any reliable vertex comes from at least a fusion of 3 frames. This consistently happens to me with both CPU and GPU. I will update the function signature by making this threshold customizable.

Can you please clarify if by CPU TSDF you mean the legacy pipeline instead of tintegrate with --device cpu:0? Otherwise it doesn't add up.

[jsaisagar]

Is there any workaround for this solution I can fix at my end?

On CPU my function is:

volume = o3d.integration.ScalableTSDFVolume( voxel_length=4.0 / 1024.0, sdf_trunc=0.04, color_type=o3d.integration.TSDFVolumeColorType.RGB8)

for i in range(1): volume.integrate( rgbd[i],intrinsics[i], poses[i])

[theNded]

I'm fixing this in a PR. Meanwhile, you can change this parameter to 0 and compile the python library if you want to temporarily fix it: https://github.com/intel-isl/Open3D/blob/master/cpp/open3d/t/geometry/kernel/GeometryMacros.h#L371

[jsaisagar]

Got the same reconstruction now