search

ialhashim / DenseDepth

High Quality Monocular Depth Estimation via Transfer Learning
https://arxiv.org/abs/1812.11941
GNU General Public License v3.0
1.59k stars 354 forks source link

Custom data #68

Closed MMehdiMousavi closed 5 years ago

MMehdiMousavi commented 5 years ago

hi,

im testing this code on my own custom data, and i wanted to ask you about a couple things:

  1. when you mention mindepth and maxdepth, what is that exactly? are you scaling the depth values in the image to a certain amount? is it a representation of range? and how are you doing these things exactly because depth in an image could only be 0 to 255? you cant have 1000 value in your pixels...?

  2. when you did train with your Unreal1k dataset, did you experience any overfitting issues and did you change things based on that? Im trying to do this myself with my custom image set, but i find that it is overfitting to training data. i tried using 500 images, 1K images, 2K images and nothing is changing. my guess is that maybe its the min max depth is a little bit different in our image sets?

appreciate the help!

ialhashim commented 5 years ago

  1. This range is to avoid the division by zero and also clip any values outside of the training range. Min depth is 0.1 meters and maximum is 10 meters for NYU dataset.

  2. I did not change the model or code for that dataset. I purposefully wanted to stress TEST the different methods on external data and the easiest way for me was to use synthetic data with perfect depth. All in all, most of the methods produce low quality results but ours seems to be the best quantitatively, meaning in an equal experiment, our model generalizes the best.

MMehdiMousavi commented 5 years ago

okay I have a follow up question: so I looked at the NYU dataset and tried to replicate the thing but in lower quantities of data. i looked at NYU about 32K but i took one virtual scene and replicated around 5000 images to look exactly like the form in NYU dataset. the model is overfitting to training data again. so did you TRAIN on your unreal dataset or did you just test on that? because i think training with 1000 images seems to be inadequate and causes overfitting?

ialhashim commented 5 years ago

The Unreal dataset is only used for testing a model trained on NYU, this gives a somewhat better understanding about how well the model perform in a different setting.

How bad of an overfitting. The described model does over fit relatively quickly on NYU. Data augmentation can help with that but in the end most DNNs overfit.

MMehdiMousavi commented 5 years ago

I reached loss of 0.05 on my training but loss is 28 on validation. also by the way, for the full training what is the .npy files that the code tries to load on utils.py? can you explain how to prepare the test set for evaluation?

MMehdiMousavi commented 4 years ago

just to clarify, i want to evaluate the model on my own test data and not the nyu one. i dont know how to construct the npy files that youre loading in evaluate.py

ialhashim commented 4 years ago

You can skip that and just take a careful look at the evaluate function (https://github.com/ialhashim/DenseDepth/blob/master/utils.py#L107).

The function takes the model, RGBs, ground truth depths, and a crop factor (if needed). Somethings that need to be adjusted is the range of values.

Keep in mind the two models trained on different ranges and sizes of images. Also one was on indoor and another is outdoor images. So whatever trained model you use you need to consider the range it was trained on and the type of scene (indoor/outdoor).