search

fabiotosi92 / monoResMatch-Tensorflow

Tensorflow implementation of monocular Residual Matching (monoResMatch) network.
116 stars 20 forks source link

Generation of proxy labels #2

Closed AI-slam closed 5 years ago

AI-slam commented 5 years ago

Hello, thanks for your sharing your work. I have some trouble to generate the proxy labes. Could you provide the corresponding scripts about how to generate it. rSGM is not sufficient. Thanks very much! @fabiotosi92

fabiotosi92 commented 5 years ago

Hi! What kind of problem did you encounter in rSGM?

AI-slam commented 5 years ago

Thanks for you reply. the riginal input of rSGM is pgm format, here, our input is grayscale or color format? Could you share the modified part based on rSGM? @fabiotosi92

mattpoggi commented 5 years ago

Hi @AI-slam, pgm format is required for 16bit images only. if you are using 8bit images, you can load any image format you desire (see rSGMCmd.cpp, line 317).

Please note that rSGM requires images having width multiple of 16. You can modify rSGMCmd.cpp adding the following code before line 324:

if (img1.cols % 16 != 0) { copyMakeBorder(img1, img1, 0, 0, 0, 16-(img1.cols % 16), BORDER_CONSTANT, 0); }

Also, remember to remove padding before saving disparities (line 386). You can convert the disparity map into a Mat structure, remove the padding and save it.

AI-slam commented 5 years ago

I try to generate the proxy as my modifies and train the network, but I can't get the expected result as your paper on just kitti eigen split. your rmse is 4.714, I am 4.973. So maybe something wrong with my proxy label.

Firstly, I read the grayscale of left and right with opencv and resize the shape into (1280, 384)(INTER_NEAREST). Then, I copy the Mat structure to MyImage structure, like MyImage MyImg1(img1.cols, img1.rows, img1.data). Lastly, I convert CV_8UC1 type to CV_16UC1, and save it as a png format after *256.0.

Well, Is there anything wrong with my handling? could you provide the definition of copyMakeBorder function, Is the padding here you mean zero value? we should set it -1 and save disparities?

Thanks for your answers again.

mattpoggi commented 5 years ago

We compute proxy labels before resizing, i.e. at KITTI original resolution, that's why we pad images to make their width multiple of 16. The resize to 1280x384 is carried out only when reading images and proxy labels in Tensorflow.

Conversion to CV_16UC1 and multiplication by *256.0 are correct.

AI-slam commented 5 years ago

Hi, I have trained your network with proxy labels, but the results is worse than yours. The parameters are same as yours. Here, abs_rel sq_rel rms log_rms a1 a2 a3 my 0.1180 0.8931 4.895 0.210 0.854 0.950 0.977 yours 0.111 0.867 4.714 0.199 0.864 0.954 0.979 I think that it is related to my generated proxy labels, so, could you provide the script of your processing? My email is super2alex1994@gmail.com, thanks a lot.

lilingge commented 3 years ago

Hi, I have trained your network with proxy labels, but the results is worse than yours. The parameters are same as yours. Here, abs_rel sq_rel rms log_rms a1 a2 a3 my 0.1180 0.8931 4.895 0.210 0.854 0.950 0.977 yours 0.111 0.867 4.714 0.199 0.864 0.954 0.979 I think that it is related to my generated proxy labels, so, could you provide the script of your processing? My email is super2alex1994@gmail.com, thanks a lot.

Hi, I generate the proxy'code as this: def generate_stereo_matchers(): numDisparity = 192 sad_window_size = 3 blockSize = 1 stereo_params = dict( preFilterCap=63, P1=sad_window_size sad_window_size 4, P2=sad_window_size sad_window_size 32, minDisparity=0, numDisparities=numDisparity, uniquenessRatio=10, speckleWindowSize=100, speckleRange=16, blockSize=blockSize) stereo_matcher = cv2.StereoSGBM_create(**stereo_params) return stereo_matcher

imL = Image.open('/opt/data/private/data/kitti/2011_09_26/2011_09_26_drive_0052_sync/image_02/data/0000000000.png') imL = np.array(imL) imR = Image.open('/opt/data/private/data/kitti/2011_09_26/2011_09_26_drive_0052_sync/image_03/data/0000000000.png') imR = np.array(imR) disp = stereo_matcher.compute(imL, imR) / 16 disp = Image.fromarray(disp) disp = disp.convert('L') disp.save('/opt/data/private/MonoDepth-PyTorch/dis_pil.png')

I didn't know whether it is correct that I generated the proxy labels in this way. Could u tell me?

lilingge commented 3 years ago

Hi, I have trained your network with proxy labels, but the results is worse than yours. The parameters are same as yours. Here, abs_rel sq_rel rms log_rms a1 a2 a3 my 0.1180 0.8931 4.895 0.210 0.854 0.950 0.977 yours 0.111 0.867 4.714 0.199 0.864 0.954 0.979 I think that it is related to my generated proxy labels, so, could you provide the script of your processing? My email is super2alex1994@gmail.com, thanks a lot.

Hi, I generate the proxy'code as this: def generate_stereo_matchers(): numDisparity = 192 sad_window_size = 3 blockSize = 1 stereo_params = dict( preFilterCap=63, P1=sad_window_size sad_window_size 4, P2=sad_window_size sad_window_size 32, minDisparity=0, numDisparities=numDisparity, uniquenessRatio=10, speckleWindowSize=100, speckleRange=16, blockSize=blockSize) stereo_matcher = cv2.StereoSGBM_create(**stereo_params) return stereo_matcher

imL = Image.open('/opt/data/private/data/kitti/2011_09_26/2011_09_26_drive_0052_sync/image_02/data/0000000000.png') imL = np.array(imL) imR = Image.open('/opt/data/private/data/kitti/2011_09_26/2011_09_26_drive_0052_sync/image_03/data/0000000000.png') imR = np.array(imR) disp = stereo_matcher.compute(imL, imR) / 16 disp = Image.fromarray(disp) disp = disp.convert('L') disp.save('/opt/data/private/MonoDepth-PyTorch/dis_pil.png')

I didn't know whether it is correct that I generated the proxy labels in this way. Could u tell me?

Hi, I have trained your network with proxy labels, but the results is worse than yours. The parameters are same as yours. Here, abs_rel sq_rel rms log_rms a1 a2 a3 my 0.1180 0.8931 4.895 0.210 0.854 0.950 0.977 yours 0.111 0.867 4.714 0.199 0.864 0.954 0.979 I think that it is related to my generated proxy labels, so, could you provide the script of your processing? My email is super2alex1994@gmail.com, thanks a lot.

Hi, I have trained your network with proxy labels, but the results is worse than yours. The parameters are same as yours. Here, abs_rel sq_rel rms log_rms a1 a2 a3 my 0.1180 0.8931 4.895 0.210 0.854 0.950 0.977 yours 0.111 0.867 4.714 0.199 0.864 0.954 0.979 I think that it is related to my generated proxy labels, so, could you provide the script of your processing? My email is super2alex1994@gmail.com, thanks a lot.

Hi, I generate the proxy'code as this: def generate_stereo_matchers(): numDisparity = 192 sad_window_size = 3 blockSize = 1 stereo_params = dict( preFilterCap=63, P1=sad_window_size sad_window_size 4, P2=sad_window_size sad_window_size 32, minDisparity=0, numDisparities=numDisparity, uniquenessRatio=10, speckleWindowSize=100, speckleRange=16, blockSize=blockSize) stereo_matcher = cv2.StereoSGBM_create(**stereo_params) return stereo_matcher

imL = Image.open('/opt/data/private/data/kitti/2011_09_26/2011_09_26_drive_0052_sync/image_02/data/0000000000.png') imL = np.array(imL) imR = Image.open('/opt/data/private/data/kitti/2011_09_26/2011_09_26_drive_0052_sync/image_03/data/0000000000.png') imR = np.array(imR) disp = stereo_matcher.compute(imL, imR) / 16 disp = Image.fromarray(disp) disp = disp.convert('L') disp.save('/opt/data/private/MonoDepth-PyTorch/dis_pil.png')

I didn't know whether it is correct that I generated the proxy labels in this way. Could u tell me?