search

shariqfarooq123 / AdaBins

Official implementation of Adabins: Depth Estimation using adaptive bins
GNU General Public License v3.0
717 stars 157 forks source link

Code changes for Uniform bin widths #41

Open koutilya-pnvr opened 3 years ago

koutilya-pnvr commented 3 years ago

How to train this model for Uniform bins sizes as reported in Table. 6 of the main paper?

The mViT outputs both bin widths and Range attention maps. So if we were to use uniform bin widths, should we just discard the mViT's predicted bin widths and recompute the bin centers on a uniform bin widths tensor and finally compute the depth prediction?

1) https://github.com/shariqfarooq123/AdaBins/blob/2fb686a66a304f0a719bc53d77412460af97fd61/models/miniViT.py#L29 The regression_head is used to feed the MLP to get the bin widths. So I am assuming we just ignore this regression_head variable and not use any MLP and just return the range_attention_maps!?

2) https://github.com/shariqfarooq123/AdaBins/blob/2fb686a66a304f0a719bc53d77412460af97fd61/train.py#L187-L195 Assuming, we did that, the bin _edges returned by the model would be a constant tensor across all the images (uniform bins constraint). Now, we shouldn't use the chamfer loss for these bins right!? because these uniform bin edges are not something that the network predicted, and moreover, we cant force these predicted bin edges to mimic the bin edges in the ground truth.

It will be great if you can share all the related changes for converting AdaBins to Uniform Bins.

1770031555 commented 2 years ago

And I have another question , if I set batch size lager than 1, then there will exists nan from the mViT I do not know the reasons , and could you please kindly help me ?