Depthstillation on real dataset

[Salvatore-tech]

Good evening, I'd like to reproduce your depthstillation process, but on real dataset called CADDY in order to evaluate some models (pwcnet and raft). I already extracted the depth maps through MIDAS and depthstilled them, but it seems that the metrics of the models that i trained on these depthstilled data are worsening a lot... I'm attaching, from top to bottom, computed flow, depth map, original and depthstilled images.

Do you see something strange in them? Any tips about the use case? Thanks in advance!

[mattpoggi]

Hi @Salvatore-tech, a few questions: 1) how many images did you use for training? 2) are all the images taken in underwater conditions? In our experience, having a large variety in terms of image context helps

[Salvatore-tech]

Hi @mattpoggi,

1. I depthstilled 4432/30000 on CADDY at the moment, because it is not a quick process with the hw i have
2. Yes all the images of the dataset are taken underwater... do i need additional preprocess to use them? (the depthstilled flow i attached above seems not showing great issues despite the underwater env)

I'm attaching a screenshot of the results with RAFT when trained on syntetic dataset and also on dCADDY and as you can see in yellow, the results are not encouraging... If we compare the first and the second raw on KITTI, the EPE and Fl are worsening a lot. Between the first and third raw on SINTEL there is no clear trend (sometimes a bit better, sometimes not).

[mattpoggi]

I'm afraid the main problem is the image content, which is very similar in all the frames used...

[Salvatore-tech]

Do you think that a feature extraction could lead to better result? I'm thinking to freeze the weights till the last layer when using the dCADDY