search

fabiotosi92 / NeRF-Supervised-Deep-Stereo

A novel paradigm for collecting and generating stereo training data using neural rendering
https://nerfstereo.github.io/
MIT License
349 stars 19 forks source link

The disparity map is not right #2

Closed liubinyijia closed 1 year ago

liubinyijia commented 1 year ago

@fabiotosi92 Hello, this project is really great! I downloaded stereo_dataset_v1_part14 and parsed the dataset from your instructions and found the disparity map is not exactly right, could you please double check the dataset?

fabiotosi92 commented 1 year ago

Hi, thank you for your positive feedback on the project. I'm sorry to hear that you're encountering an issue with the stereo_dataset_v1_part14. Could you please provide more details about the specific problem you're facing with the dataset?

liubinyijia commented 1 year ago

@fabiotosi92 Hi, this is the test code, and the yellow point in the visualization result is one of the the corresponding pair , please have a look!

` import cv2 import numpy as np

imgl = cv2.imread('stereo_dataset_v1_part14/0260/Q/baseline_0.10/left/IMG_20221007_113431_TIMEBURST1.jpg') imgr = cv2.imread('stereo_dataset_v1_part14/0260/Q/baseline_0.10/right/IMG_20221007_113431_TIMEBURST1.jpg') disp = cv2.imread('stereo_dataset_v1_part14/0260/Q/baseline_0.10/disparity/IMG_20221007_113431_TIMEBURST1.png', cv2.IMREAD_ANYDEPTH) disp = disp/64.0

vispos = (84, 660) #y,x disp_offset = disp[vispos[0], vispos[1]] print (disp_offset)

cv2.circle(imgl, (vispos[1], vispos[0]), 5, (0, 255, 255) , 2) cv2.circle(imgr, (int(vispos[1]-disp_offset), vispos[0]), 5, (0, 255, 255) , 2) visImg = np.hstack((imgl, imgr)) cv2.imshow('vis', visImg) cv2.waitKey(0) `

vis

fabiotosi92 commented 1 year ago

@liubinyijia It seems that you are using the disparity maps incorrectly in your code. Specifically, all the disparity maps in the dataset are aligned with the 'center' image, but it appears that you are mistakenly using the 'left' image as the reference.

To resolve this, please replace the following line in your code: imgl = cv2.imread('stereo_dataset_v1_part14/0260/Q/baseline_0.10/left/IMG_20221007_113431_TIMEBURST1.jpg')

with this line: imgl = cv2.imread('stereo_dataset_v1_part14/0260/Q/center/IMG_20221007_113431_TIMEBURST1.jpg')

Please note that our dataset provides image triplets characterized by 'left,' 'center,' and 'right' viewpoints. The 'center' image should be used as the reference image. By aligning the disparity map with the 'center' image, you will achieve accurate results.

Furthermore, it's important to note that the raw disparity maps computed by InstantNGP can be quite noisy. It would be advisable to apply filtering using the confidence encoded in the available Ambient Occlusion (AO) maps. This should help improve the accuracy of your disparity check experiment.

Please give these adjustments a try, and let me know if you encounter any further issues.