search

andyzeng / tsdf-fusion

Fuse multiple depth frames into a TSDF voxel volume.
http://andyzeng.github.io/
BSD 2-Clause "Simplified" License
713 stars 133 forks source link

question about projection of 3d point generated from TSDF #8

Open Kyungdon opened 6 years ago

Kyungdon commented 6 years ago

Hi @andyzeng I'm trying to use your code (tsdf2mesh.m function) to generate the 3d point cloud. From MSR 7-scene dataset, when I project the 3d point cloud generated by your function into image domain, it shows some misalignment. It could be my mistake (I considered extrinsic pose between depth and color camera).

Have you tried your function for MSR 7-scene dataset? or any suggestion for this kind of procedure?

andyzeng commented 6 years ago

Hello! Yes, the function should work for the MSR 7-scene dataset. In fact, the RGB-D frames used in the demo code is actually from the redkitchen sequence from the 7-scene dataset :)

Personally, I have not tried back-projecting the 3D point cloud back onto the image domain. However, upon revisiting tsdf2mesh.m, I noticed a bug where the mesh being saved was still in voxel coordinates rather than camera coordinates. I've pushed a fix for this in the latest commit. I wonder if that will fix your problem.

Kyungdon commented 6 years ago

Thanks for quick reply, @andyzeng Still, there is a few issues.

1) binary file of MSR 7-scene dataset does not have "tsdfHeader". Thus I just got the value from *.mhd, but still, it doesn't have truncMargin, etc. and your code does not use truncMargin.

2) With some modification, I could align them roughly but it still has misalignment (please see attached image file, red point cloud from tsdf, other color points from depth. you can see some misalignment at the pumpkin). misalign

Do you have any advice to figure this issue out?

andyzeng commented 6 years ago

I see, I had misunderstood your original question. You were asking about whether the tsdf2mesh function works for the binary voxel volume files of the MSR 7-scene dataset. I haven't tried that myself, so I don't know all of the details behind setting it up.

Although, from looking at your misaligned point cloud, I'm guessing that the problem is in your camera pose, or voxel volume offset. From the description on the dataset's website, they quote "The spacings and offsets in millimeters are provided in the header file." Does your current implementation use those offsets?

Kyungdon commented 6 years ago

Yes, I'm using the offsets in millimeters.