jonbarron
commented
4 years ago Cool, I've gotten many requests for this SFMLearner result in Pytorch but I don't have cycles to take care of it myself, so I'm happy to help.
Here's the part of the main paper that describes important changes: We keep our loss’s scale c fixed to 0.01, thereby matching the fixed scale assumption of the baseline model and roughly matching the shape of its L1 loss (Eq. 15). To avoid exploding gradients we multiply the loss being minimized by c, thereby bounding gradient magnitudes by residual magnitudes (Eq. 14).
Section H of the appendix goes through all the changes that were made to the codebase:
- Our unsupervised monocular depth estimation experiments use the code from https://github.com/tinghuiz/ https://github.com/tinghuiz/ SfMLearner, which appears to correspond to the “Ours (w/o explainability)” model from Table 1 of [42]. The only changes we made to this code were: replacing its loss function with our own, reducing the number of training iterations from 200000 to 100000 (training converges fast when using our loss function) and disabling the smoothness term and multi-scale side predictions used by [42], as neither yielded much benefit when combined with our new loss function and they complicated experimentation by introducing hyperparameters. Because the reconstruction loss in [42] is the sum of the means of the losses imposed at each scale in a D-level pyramid of side predictions, we use a D level normalized wavelet decomposition (wherein images in [0, 1] result in wavelet coefficients in [0, 1]) and then scale each coefficient’s loss by 2 d , where d is the coefficients level.*
So with this, looking at the code you sent me, It looks like you should do: 0.01 * AdaptiveImageLossFunction((3, 640, 192), torch.float32, "cuda:0", color_space='YUV', scale_lo=0.01, scale_init=0.01, wavelet_scale_base=$SOMETHING) And then just set the final loss to be 100% this adaptive loss (no SSIM or smoothness). I think wavelet_scale_base should be 2, but the interface for that experiment isn't the same as what is here so try [0.5, 1, 2]. And be sure to delete the secondary branches of the code that evaluate this loss at multiple scales, you should only need to evaluate it at the finest scale.
On Mon, Mar 9, 2020 at 3:44 AM Varun Ravikumar notifications@github.com wrote:
Hi, unfortunately we don't have a code release for the monocular depth estimation experiments of the paper (though that code is in TF anyways so it likely isn't what you're looking for). I believe that there are Pytorch implementations of SFMLearner on Github, and using this loss should be straightforward: just delete the existing multiscale photometric loss and the smoothness term and add in AdaptiveImageLossFunction on the full-res image with: scale_lo=0.01 scale_init=0.01 and default settings for the rest and it should work (you may need to fiddle with the value of wavelet_scale_base).
Do you suggest any other changes apart from this? I am actually testing this :)
Tested your loss in YUV space on monodepth2 project. AdaptiveImageLossFunction((3, 640, 192), torch.float32, "cuda:0", color_space='YUV', scale_lo=0.01, scale_init=0.01) These were the initial setting used. Actually, the scale of final loss of monodepth with L1 is around 0.02. After adding your loss it just drives it below 0. Final loss = 0.85 Adaptive Loss + 0.15 SSIM + 0.01 Smoothness
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/jonbarron/robust_loss_pytorch/issues/16?email_source=notifications&email_token=AAGZFNQEYEXADKKEJG3TDFDRGTCANA5CNFSM4LEF3YD2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4ITQXSCA, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGZFNSV2JT4IWW3LL4ZDW3RGTCANANCNFSM4LEF3YDQ .
rvarun7777
yellowYuga
Hi, unfortunately we don't have a code release for the monocular depth estimation experiments of the paper (though that code is in TF anyways so it likely isn't what you're looking for). I believe that there are Pytorch implementations of SFMLearner on Github, and using this loss should be straightforward: just delete the existing multiscale photometric loss and the smoothness term and add in AdaptiveImageLossFunction on the full-res image with: scale_lo=0.01 scale_init=0.01 and default settings for the rest and it should work (you may need to fiddle with the value of wavelet_scale_base).
Do you suggest any other changes apart from this? I am actually testing this :)
If you call the loss in general.py, it should never go negative. The loss produced by adaptive.py is actually a negative log-likelihood, so it can go negative depending on the value of the scale parameter. Could you kindly elaborate on this? Doesn't the -log(y) yield positive values. Example: NLL: -ln(0.5) = 0.69 How does it drive negative or do we consider the abs(l1) finally?
After adding your loss it just drives it below 0 and the training fails.
Tested your loss in YUV space on monodepth2 project. AdaptiveImageLossFunction((3, 640, 192), torch.float32, "cuda:0", color_space='YUV', scale_lo=0.01, scale_init=0.01) Note: Alpha and Scale are added to the optimizers. I have skipped this part here. These were the initial settings used. Actually, the scale of final loss of monodepth with L1 is around 0.07 to 0.02 and performs well.
Test 1 Final loss = 0.85 Adaptive Loss + 0.15 SSIM + 0.001 Smoothness Test 2: Tried different weightage as well. This setting would fail after few epochs as well. Final loss = 0.15 Adaptive Loss + 0.85 SSIM + 0.001 Smoothness