search

shichaoy / semantic_3d_mapping

Semantic 3D Occupancy Mapping through Efficient High Order CRFs
Other
193 stars 60 forks source link

There are something wrong with me when I test on the NYUv2 dataset. #5

Open myhooo opened 5 years ago

myhooo commented 5 years ago

Hi, thanks for your share!

I am testing your code on the NYUv2 dataset. ^ _ ^

I have used your code in folder named 'preproess_data/superpixel/' and have made the image like this image Then transform the image to bin using your code.

I make the semantic segmentation using the FCN, in which there is model on the dataset. The segmentation image like this image Then translate the image into bin using the code: net.blobs['prob'].data[0].transpose([1,2,0]).flatten().tofile('/home/x/bin/nyu'+ str(i) + '.bin')

And I get the pose through the ORB-SLAM2

Then I run the mapping_3d, there isn't any warning and error. image

However, the result seems to be wrong. -_- The image produced in the folder named 'crf_3d_reproj' seems to be wrong, like this image

As a result, the semantic 3d mapping looks bad and like this image

Could you please help me and tell me what the possible reasons are ?

Thanks!

shichaoy commented 5 years ago

Hi, thanks for testing. I cannot root the problem right now. There might be some parameters tuned for outdoor KITTI, which may not be suitable for small scale indoors. For example, grid resolution, size, CRF position kernel std, weight. Before tuning params, make sure to give correct calibration and image size in the main file. Then do experiments step by step. First verify whether geometric 3D mapping is good or not. Then without CRF optimization (only label back-projection). Then with CRF, but no hierarchical. Finally with hierarchical. I look through the code and seems to find a bug in the main file: I set curr_transToWolrd(2,3)=1.0 when saving image. Can you also try commenting it.

shichaoy commented 5 years ago

just let me know if you run more small tests as I said. at that time, I may have more clues what's wrong.

mengqiguo commented 1 month ago

I had a similar problem to yours, did you find a solution?