search

LiheYoung / Depth-Anything

[CVPR 2024] Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data. Foundation Model for Monocular Depth Estimation
https://depth-anything.github.io
Apache License 2.0
6.39k stars 491 forks source link

How train model on other dataset with satellite images? #123

Open ValeriaIM opened 3 months ago

ValeriaIM commented 3 months ago

Hello, I'm interested in training your model on satellite images. Is this even possible? Or is the network created only for pictures of streets and premises?

If this is possible, please tell me how best to do this

LiheYoung commented 3 months ago

Hi,

  1. If you hope to train a relative disparity model, similar to Depth Anything, you are suggested to load the entire Depth Anything pre-trained parameters (including the encoder and decoder) for fine-tuning with your own satellite images.
  2. If you hope to train a metric depth model, you are suggested to load the pre-trained encoder parameters of Depth Anything, and randomly initialize the decoder for fine-tuning with your own satellite images. We also use this principle to fine-tune foundation Depth Anything models for metric depth estimation. Please refer to here for details.
ValeriaIM commented 3 months ago

So if I need a metric depth model, then I should try training a zoedepth model with your checkpoints? Or use the model that is in the link? how will this model differ from the zoedepth model internally?

And how to initialize the decoder for fine tuning? In which file and line should this be done?

nabinanaa commented 3 months ago

hi！i am also trying to train one depth model on satellite images, can we have a discussion about this?

ValeriaIM commented 3 months ago

Hi, yes, of course. Now i work with the network ZoeDepth (network Depth-Anything based on this) and use URBAN SEMANTIC 3D DATASET. I need to train ZoeDepth to get depth using only one image. To achieve this, i've written code for image processing for DataLoader, added config and changed the loss function (= 1 - r2). Now i'm also working with the network Depth Anything, but there are few results

roman-studer commented 3 days ago

Hi all. Did you manage to get some good results when training on satellite data? And do you know of any avail. datasets that suite this usecase? I'm currently planning a project that requires high quality depth maps of buildings. The base model seems promising , and is useable for single buildings. But the height difference between neighbouring buildings is often not right.

Kind regards!