Crash due to fast motion?

[bsespede]

Hello, I've noticed that the algorithm simply crashes upon frame with "fast" motion from one frame to the other. The reconstruction up to that point looks pretty much perfect. In particular, the program reports:

_... /home/bsespede/Repos/surfelwarp/apps/surfelwarp_app/main.cpp:40: The 39th Frame /home/bsespede/Repos/surfelwarp/core/warp_solver/DensityForegroundJacobianResidual.cu:192: There is no foreground pixel /home/bsespede/Repos/surfelwarp/core/warp_solver/DensityForegroundJacobianResidual.cu:192: There is no foreground pixel /home/bsespede/Repos/surfelwarp/core/warp_solver/DensityForegroundJacobianResidual.cu:192: There is no foreground pixel /home/bsespede/Repos/surfelwarp/core/warp_solver/DensityForegroundJacobianResidual.cu:192: There is no foreground pixel /home/bsespede/Repos/surfelwarp/core/warp_solver/DensityForegroundJacobianResidual.cu:192: There is no foreground pixel /home/bsespede/Repos/surfelwarp/core/warpsolver/DensityForegroundJacobianResidual.cu:192: There is no foreground pixel Error: /home/bsespede/Repos/surfelwarp/core/geometry/DoubleBufferCompactor.cu:165

Is this an error that is to be expected? Is it possible to somehow recover from bad frames?

Best regards, Braulio

[weigao95]

The message "there is no foreground pixel" means either 1) The clipping distance is wrong. The object for reconstruction is not within the specified min and max distance. 2) The foreground segmentation reports no foreground.

Please first try to modify the clipping distance in the config file and have a try.

[bsespede]

Could this potentially cause the algorithm to crash? I've managed to make other reconstructions with this warning and they seemed ok. Furthermore, I also experimented settings smaller and larger clipping ranges and it would still crash with the error: /home/bsespede/Repos/surfelwarp/core/geometry/DoubleBufferCompactor.cu:165

[xiaotaw]

I also met the same problem. In #33 , when camera moves fast, surfelwarp_app may crash down and report the same error.

@bsespede did you record the color and depth images that can reproduce the problem?

[weigao95]

Could this potentially cause the algorithm to crash? I've managed to make other reconstructions with this warning and they seemed ok. Furthermore, I also experimented settings smaller and larger clipping ranges and it would still crash with the error: /home/bsespede/Repos/surfelwarp/core/geometry/DoubleBufferCompactor.cu:165

Could you provide more details regarding the error in DoubleBufferCompactor? Is that because the array to compact from is empty?

[bsespede]

Hi,

@xiaotaw I dont have the same scene where it crashed recorded but I have another one with the same error. Its using the VolumeDeform dataset format so it should be plug-and-play to test it out: https://drive.google.com/file/d/1WMol0P4jM49h22bLlCZn37w_-NA4dH03/view?usp=sharing

@weigao95 Commenting out the lines 183 and 184 in DoubleBufferCompactor.cu, I get the following output on the above dataset:

_... /home/bsespede/Repos/surfelwarp/core/geometry/DoubleBufferCompactor.cu:183: The number of remaining surfel is 123894 while the size of original surfel is 123894 /home/bsespede/Repos/surfelwarp/core/geometry/DoubleBufferCompactor.cu:184: The number of appended surfel is 3543 /home/bsespede/Repos/surfelwarp/core/geometry/WarpFieldUpdater.cpp:162: Invalid wight in node KNN /home/bsespede/Repos/surfelwarp/core/geometry/WarpFieldUpdater.cpp:162: Invalid wight in node KNN /home/bsespede/Repos/surfelwarp/apps/surfelwarp_app/main.cpp:40: The 23th Frame /home/bsespede/Repos/surfelwarp/core/geometry/DoubleBufferCompactor.cu:183: The number of remaining surfel is 127437 while the size of original surfel is 127437 /home/bsespede/Repos/surfelwarp/core/geometry/DoubleBufferCompactor.cu:184: The number of appended surfel is 70 /home/bsespede/Repos/surfelwarp/apps/surfelwarp_app/main.cpp:40: The 24th Frame /home/bsespede/Repos/surfelwarp/core/geometry/DoubleBufferCompactor.cu:183: The number of remaining surfel is 127507 while the size of original surfel is 127507 /home/bsespede/Repos/surfelwarp/core/geometry/DoubleBufferCompactor.cu:184: The number of appended surfel is 178 /home/bsespede/Repos/surfelwarp/apps/surfelwarp_app/main.cpp:40: The 25th Frame /home/bsespede/Repos/surfelwarp/core/geometry/DoubleBufferCompactor.cu:183: The number of remaining surfel is 127685 while the size of original surfel is 127685 /home/bsespede/Repos/surfelwarp/core/geometry/DoubleBufferCompactor.cu:184: The number of appended surfel is 834 /home/bsespede/Repos/surfelwarp/apps/surfelwarpapp/main.cpp:40: The 26th Frame Error: /home/bsespede/Repos/surfelwarp/core/geometry/DoubleBufferCompactor.cu:165

The exported point cloud right before the crash seems to be ok.

[weigao95]

Can you enable the CUDA_DEBUG_SYNC_CHECK in common/global_configs.h and check the error again?

[bsespede]

Now its crashing in SparseCorrespondenceHandler.cu with the following error: /home/bsespede/Repos/surfelwarp/apps/surfelwarp_app/main.cpp:40: The 26th Frame Error: /home/bsespede/Repos/surfelwarp/core/warp_solver/SparseCorrespondenceHandler.cu:272

I've also noticed that sometimes it also crashes with another error, but I guess its unrelated to the previous one: /home/bsespede/Repos/surfelwarp/apps/surfelwarp_app/main.cpp:40: The 20th Frame /home/bsespede/Repos/surfelwarp/core/geometry/WarpFieldUpdater.cpp:21: Nan in node candidate terminate called after throwing an instance of 'std::runtime_error*'

[xiaotaw]

@bsespede, It took me some days to download the dataset, due to the poor network.

Using your data and with CUDA_DEBUG_SYNC_CHECK, the result is: /home/xt/Documents/data/surfelwarp/apps/surfelwarp_app/main.cpp:28: The 0th Frame /home/xt/Documents/data/surfelwarp/apps/surfelwarp_app/main.cpp:28: The 1th Frame Error: /home/xt/Documents/data/surfelwarp/imgproc/normal_map.cu:124

I find cuda-programing interesting, and will try to find the bug out, in the next few days, em, maybe weeks.

[weigao95]

Now its crashing in SparseCorrespondenceHandler.cu with the following error: /home/bsespede/Repos/surfelwarp/apps/surfelwarp_app/main.cpp:40: The 26th Frame Error: /home/bsespede/Repos/surfelwarp/core/warp_solver/SparseCorrespondenceHandler.cu:272

I've also noticed that sometimes it also crashes with another error, but I guess its unrelated to the previous one: /home/bsespede/Repos/surfelwarp/apps/surfelwarp_app/main.cpp:40: The 20th Frame /home/bsespede/Repos/surfelwarp/core/geometry/WarpFieldUpdater.cpp:21: Nan in node candidate terminate called after throwing an instance of 'std::runtime_error*'

I guess the problem is that the sparse correspondence finder cannot establish correct correspondence. Can you check the correspondence result of image processor using {project}/apps/imgproc_test? You might need DrawImagePairCorrespondence function in Visualization namespace.

The correspondence is an implementation of global patch collider, and might need additional training for good performance.