search

mihaibujanca / dynamicfusion

Implementation of Newcombe et al. CVPR 2015 DynamicFusion paper
BSD 3-Clause "New" or "Revised" License
394 stars 105 forks source link

Model geometry matching prediction #49

Open Smallha61109 opened 6 years ago

Smallha61109 commented 6 years ago

Hi @mihaibujanca , I have been checking on the part of surface fusion (kinect fusion's part) of this project. I found that if I turn off the non-rigid warping part and use rigid data instead, the estimation of the model geometry matching from the ICP (or maybe from other part i'm not sure) seems to be wrong. The geometry matching failed so that the surface fusion is fusion on the wrong surface. Maybe you can check this out? Thank you very much!

mihaibujanca commented 6 years ago

Hi, thanks for the issue!

Just to make sure I understand, do you mean that you tried turning off the non-rigid part, ran it on rigid data and it failed?

If so, I'll look into it soon, I didn't expect that

Smallha61109 commented 6 years ago

Yes, what I did is I comment line 385 to 389 in kinfu.cpp (the warping functions). I think this left only surface fusion working and so I tested it with rigid data from https://rgbd-dataset.cs.washington.edu/dataset/rgbd-scenes/ . But the reconstruction is not very well.

I believe the ICP part have some problem as I lower the threshold and increased the iterations to more then ten thousand times simultaneously trying to get a better surface matching result. However, I keep getting a reset on every frame (which means ICP can't get a proper estimate on surface matching prediction).

mihaibujanca commented 6 years ago

I'll try this myself when I get home but what I'd probably do instead is replace dynamicfusion(d, pts, n); on line 225 with volume_->integrate(dists_, poses_.back(), p.intr);

Smallha61109 commented 6 years ago

I've tried what you suggested but it seems like in some cases there are still miss match between models so the reconstruction is no very well. Also another issue is that if ICP keeps failing and causing a reset, cuda will eventually run out of memory and results in a crash.

mihaibujanca commented 6 years ago

Yeah, makes sense. I'll try to see if I can come up with a quickfix for it soon - I feel like it's something minor I might have changed, since I started from a kinectfusion implementation that worked. Hopefully in the process I'll get to write some tests as well.

If it's not a minor thing, it will take a bit to solve since I'm just starting my exams period