A confusing problem about the function named _downsample in wavelet.py

[TengFeiHan0]

Goal: I trying to reproducing your implementation about SfMLearner

here are my modifications as below: adaptive= robust_loss_pytorch.adaptive.AdaptiveImageLossFunction((640,192,3), float_dtype=np.float32, device='cpu')

l1_loss =torch.mean(adaptive.lossfun(torch.abs(target - pred))[:, None]) Then I got a confusing error,

robust_loss_pytorch/wavelet.py", line 209, in _downsample
    stride=stride)[:, 0, :, :]
RuntimeError: Calculated padded input size per channel: (6 x 48). Kernel size: (7 x 1). Kernel size can't be greater than actual input size

Here is that line where we got the above error

 y = torch.conv2d(x_padded[:, np.newaxis, :, :],f_ex[np.newaxis, np.newaxis].type(x.dtype),
      stride=stride)[:, 0, :, :]

Question: where could I change the kernel_size? or how to fix this error? Is there any error with my modifications? Thank you in advance.

[jonbarron]

It's hard to say without looking at all of your code, but the error produced by the wavelet code might be due to the number of "levels" in the wavelet decomposition being too high for the image resolution. This should be something checked for in get_max_num_levels() though, so I'm not sure why it would be happening. Try decreasing wavelet_num_levels to 4 or 3 and see if that gets rid of the error (if you have to decrease it to 1 or 0 to get things to compile then something is very wrong, so don't do that). If that doesn't work, try setting representation to 'DCT' as that should produce comparable results and avoids the wavelet code that appears to be throwing this error.

That being said, the default number of levels in the wavelet decomposition should be small enough that you shouldn't be getting this error (it's designed for 64x64 images which is smaller than yours). Is it possible that you're calling the loss with reshaped or transposed dimensions? Could it be that the batch size is being interpreted as a spatial dimension?

On Mon, Dec 30, 2019 at 11:06 PM Tengfei han notifications@github.com wrote:

Goal: I trying to reproducing your implementation about SfMLearner

here are my modifications as below: adaptive= robust_loss_pytorch.adaptive.AdaptiveImageLossFunction((640,192,3), float_dtype=np.float32, device='cpu')

l1_loss =torch.mean(adaptive.lossfun(torch.abs(target - pred))[:, None]) Then I got a confusing error,

robust_loss_pytorch/wavelet.py", line 209, in _downsample stride=stride)[:, 0, :, :] RuntimeError: Calculated padded input size per channel: (6 x 48). Kernel size: (7 x 1). Kernel size can't be greater than actual input size

Here is that line where we got the above error

y = torch.conv2d(x_padded[:, np.newaxis, :, :],f_ex[np.newaxis, np.newaxis].type(x.dtype), stride=stride)[:, 0, :, :]

Question: where could I change the kernel_size? or how to fix this error? Is there any error with my modifications? Thank you in advance.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/jonbarron/robust_loss_pytorch/issues/12?email_source=notifications&email_token=AAGZFNUAVY676C7XZKRIFJDQ3LVPPA5CNFSM4KBTGKM2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IDOOTSQ, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGZFNVVBVNF7REBT5Y7ZZTQ3LVPPANCNFSM4KBTGKMQ .

[TengFeiHan0]

Yes!!! you're right ! ! !, before got the above error, I annotated this line " assert max_num_levels >= num_levels" in a func named contruct.

The reason why I ignored the assertion line:

get_max_num_levels() returns 8 while the default setting is 5, it's surprising that I encountered another error like this

 assert max_num_levels >= num_levels
AssertionError

so I annotated this line and then got the previous error.

I followed your instructions that decreasing num_levels to 3, but the error still exists. Then I changed the representation into 'DCT', fortunately, my program works well. Thank you for your kindness and rapid response. I am so sorry to trouble you on such a beautiful day. I hope you will have a nice holiday and enjoy your time.@jonbarron

[jonbarron]

I thought about this some more and I think I understand why you were having this problem. SFMLearner works by constructing multiple losses on the image at different scales. I think you were having this problem because the default value of num_levels was too large for some of the smaller scales of this multi-scale loss.

In my experiments, I found that the performance of SFMLearner could be improved substantially by deleting its multi-scale losses, and instead just using an adaptive+wavelet loss on the full resolution image. I would strongly encourage you to do the same. You can control the influence of the loss at each scale by modifying the wavelet_scale_base parameter. This requires using the wavelet version of the loss, as this control over each scale is not possible using the DCT version. This is discussed in more detail in Appendix H of the paper.

Please let me know if you're able to reproduce the results in the paper and share your code, as I think the community would greatly appreciate a Pytorch implementation of that experiment (I've been meaning to release my TF code but I haven't been able to find the time).