search

calpoly-csai / argo-editor

BSD 3-Clause "New" or "Revised" License
2 stars 0 forks source link

Test Depth Estimation Accuracy on Panoramic Images #6

Open Waidhoferj opened 3 years ago

Waidhoferj commented 3 years ago

Our depth estimation relies on the out-of-the-box implementation of the MiDaS model. MiDaS works well on standard aspect-ratio images, but we need to ensure it works for panoramic data.

Suggestions:

  1. Find a dataset for panoramic imagery with accurate depth representations. Here is an example to look into.
  2. Feed one of these images into the MiDaS model. Feel free to use the example MiDaS Notebook I built.
  3. Describe model accuracy with a loss function.
  4. Report back with your findings (in the comments below)
  5. Mess around with transfer learning and try to improve the default model accuracy.
nairrrahul commented 3 years ago

Tried out MiDaS with 10 images from this dataset Used mean squared error for loss calculations Original Image and Corresponding "Error Map" for 10 images, white -> black is more -> less error the main sources for error are generally foliage, detail in the distance, or distortion from the sides of the image Colab (.npy files taken from /val/depth in dataset)

richagadgil commented 3 years ago

Wanted to add an extra, optional step: Assessing how well each model works with our test set of Cal Poly panoramas.