About the pose in data-set

[RouPeng]

Hi, Recently i am using your frame to train with my own data-set, but i am confused about the pose in the data-set, do i need to get the pose in advance or if there are any other ways that can get the pose automatically in your frame?

[andyzeng]

Hello! If you're training with your own RGB-D frames, you will need to get the camera poses in advance - typically from a 3D reconstruction algorithm of your choice (e.g. KinectFusion).

[RouPeng]

Hi andyzeng！ Thank you for your reply！ I've prepared the data-set and tried to train the net, but every training session will stop at the end of first iteration, like this:

The state shown in the picture will remain unchanged until I stop this progress. The picture resolution in my data set is 640*480, and the pose has been calculated in advance. I want to know what else should I pay attention to, in order to avoid the failure of the training?

[andyzeng]

The data layer (see match.hpp) in the network tries to find correspondences on-the-fly during training. So one possible reason for why it may be freezing is that it fails to find correspondences with your dataset and camera poses. I would double check your camera poses by making sure that you can produce a reasonable 3D reconstruction of the scene with your RGB-D data.

Another thing to keep in mind is the format of the depth images, which can differ between datasets. 3DMatch reads them in 16-bit format in millimeters (see ReadDepth here). It's possible that the data layer cannot find correspondences because of failing to depth data properly.