search

nianticlabs / monodepth2

[ICCV 2019] Monocular depth estimation from a single image
Other
4.13k stars 953 forks source link

White predictive mask after several iterations #454

Open jordigc2 opened 2 years ago

jordigc2 commented 2 years ago

Hello,

Thank you very much for the great work that you have shared with the community. I am trying to train the model using my own dataset and I am struggling a bit to understand how the predictive mask works as it's making the network learn way better than using the automasking. However, when I check on tensorboard the output of the predictive mask it's all white. Which means that the predicted output is full of 1s so it's not playing a role in the training.

I have also seen this old issue #64 where you say that during the first iterations the predictive mask was also white for them but after it started to show some interesting results. I have trained monodepth for 30 epoch and 40K steps and the predictive mask remained white.

I also attach some images of the dataset that I am using(stereo training using ZED2 camera with a baseline of 12cm). As you can see, both depths are coherent with the input image and also some comparison has been done with the ZED2 camera depth and it's also quite close to it. However, it would be interesting to understand if this improvement is due to not using any mask, as in this case the predicted depth is full of 1s or if it really has an effect to the output. Also, the predicted mask would be useful to use in a future to identify areas that are not of interested from the depth image.

depth_estimation2 Depth_prediction

Here you can see an output of the depth using the automask, where you can see that the predictions are not as sharp and even the car is not being detected/ignored properly, as the depth value is set to be zero. depth_estimation_nopredmask

For the training where I am using the predictive mask I am also using the avg_reprojection loss flag. Could this influence on the results of the predictive mask?

Thank you very much for your time.

Jordi

9796l commented 1 year ago

hello,excuse me ,i also use my dataset ,i have a question,i am confused about K intrinsic matrix,i need to use my dataset,so i want to change K.Would you like to talk with me ?Thank you very much.