KFusion depth image format

[swarth100]

Hello Mihai,

Over the past few days we've had severe issues when using custom data not logged from a Kinect camera.

I was wondering if you just could provide a tiny extra bit of insight on the format.

Our current assumptions are the following: 1) Clipping depth is 1m. 2) Anything beyond 1m has depth value of 0. 3) Closest depth value is 2^16. 3) Anything between 0m-1m has a depth value scaled from 2^0 and 2^16.

At this link, you can find a few frames of the example data we're using.

The error the library throws is the following:

Error is not thrown with custom data.

All we've been able to trace it down to is cloud size. Cloud size for our custom input seems to always be 0. When using the example (umbrella) data we get a cloud size of 48173.

Hope you can provide further insight into this issue,

Alberto

[swarth100]

For anyone having this issue, make sure your camera treats depth values beyond the clipping plane properly.

What this library actually expects is:

Depth value 0 = closest to camera
Depth value (max uint16_t) = furthest away from camera on the horizon.

The above assumptions (2) and (3) are to be violated

[swarth100]

Actually setting too many pixels to (max uint16_t) will also crash the library.

We're working on a better estimate, yet so far we can only get frames to run when ~80% of the image has actually mapped depth data.

We're working on seeing if scaling the max depth values has any effect

[mihaibujanca]

What's the error it's crashing with?

[Db2515]

I am working with swarth100 and we are still having this problem. We get the same error as above: KinFu2 error: invalid configuration argument /homes/db2515/Documents/Programming/DynamicFusionCore/kfusion/src/cuda/tsdf_volume.cu:831

Initially we were setting any 0 values to (max uint16_t) with the aim to remove pixels past a certain distance when loading the data into your library. Since then we have noticed that the truncate method appears set any values above the truncation distance to 0 so we have left the values at 0 during logging but still get the same issues that if to many values are 0 we get this error. Setting the p.icp_truncate_depth_dist values to something other than 0 doesn't appear to help either.

[mihaibujanca]

Any chance you could post the stack trace?

[Db2515]

The stack trace we are getting is: Cuda API error detected: cudaLaunch returned (0x9) (cuda-gdb) where

#0  0x00007fffdb43c070 in cudbgReportDriverApiError () from /usr/lib/x86_64-linux-gnu/libcuda.so.1
#1  0x00007fffdb43ed76 in cudbgReportDriverInternalError () from /usr/lib/x86_64-linux-gnu/libcuda.so.1
#2  0x00007fffdb441de4 in cudbgApiDetach () from /usr/lib/x86_64-linux-gnu/libcuda.so.1
#3  0x00007fffdb629753 in cudbgMain () from /usr/lib/x86_64-linux-gnu/libcuda.so.1
#4  0x00007fffdb636753 in cudbgMain () from /usr/lib/x86_64-linux-gnu/libcuda.so.1
#5  0x00007ffff7ba47be in cudaLaunch () from /vol/cuda/8.0.44/lib64/libcudart.so.8.0
#6  0x0000000001df4ba4 in cudaLaunch<char> (
    func=0x1df4853 <kfusion::device::extract_normals_kernel(kfusion::device::ExtractNormals, float4*)> "UH\211\345H\203\354\020H\211}\370H\213E\370H\211\306H\215}\020\350i\377\377\377\220\311\303UH\211\345H\203\354\020H\211}\370H\213E\370H\211\005\av\026\001H\213E\370H\211\307\350\360\343\377\377H\213E\370j") at /vol/cuda/8.0.44/include/cuda_runtime.h:1819
#7  0x0000000001df484b in __device_stub__ZN7kfusion6device22extract_normals_kernelENS0_14ExtractNormalsEP6float4 (__par0=..., __par1=0x0)
    at /tmp/tmpxft_000b4d96_00000000-4_tsdf_volume.compute_61.cudafe1.stub.c:1
#8  0x0000000001df486f in kfusion::device::extract_normals_kernel (__cuda_0=..., __cuda_1=0x0) at /homes/db2515/Documents/Programming/DynamicFusionCore/kfusion/src/cuda/tsdf_volume.cu:795
#9  0x0000000001df424e in kfusion::device::extractNormals (volume=..., points=..., aff=..., Rinv=..., gradient_delta_factor=0.5, output=0x0)
    at /homes/db2515/Documents/Programming/DynamicFusionCore/kfusion/src/cuda/tsdf_volume.cu:830
#10 0x0000000001decb46 in kfusion::cuda::TsdfVolume::fetchNormals(kfusion::cuda::DeviceArray<kfusion::Point> const&, kfusion::cuda::DeviceArray<kfusion::Point>&) const ()
#11 0x0000000001ded5c6 in kfusion::cuda::TsdfVolume::compute_normals() ()
#12 0x0000000001dc6048 in kfusion::KinFu::operator()(kfusion::cuda::DeviceArray2D<unsigned short> const&, kfusion::cuda::DeviceArray2D<kfusion::RGB> const&) ()
#13 0x0000000001dd08da in DynamicFusionApp::execute() ()
#14 0x000000000044b893 in main ()

[mihaibujanca]

It's quite hard to figure this out, but two things I did notice:

cudbgReportDriverApiError and cudbgReportDriverInternalError suggest there is some kind of incompatibility with the driver. Could you confirm what CUDA version are you running and on what GPU?

#9  0x0000000001df424e in kfusion::device::extractNormals (volume=..., points=..., aff=..., Rinv=..., gradient_delta_factor=0.5, output=0x0)

output=0x0 looks like a null pointer to me, so that might have something to do with it.

I'll investigate further and let you know