Closed wyli closed 1 year ago
perhaps with large kernels, there's a problem with the zero padding here?https://github.com/Project-MONAI/MONAI/blob/817c1cf1648417ed2688d7f9f8c01f0cb3eecbcf/monai/networks/layers/simplelayers.py#L208 @ebrahimebrahim
Copying this from the discussion: I have a notebook demonstrating the issue: https://github.com/ebrahimebrahim/deep-atlas/blob/main/lncc_with_large_kernel.ipynb
perhaps with large kernels, there's a problem with the zero padding here?
:thinking:
thanks for the detailed demo, I did a quick change of the mode zeros
to circular
in separable_filtering
~it seems the loss becomes more reasonable~ grad size still doesn't look right.
I would be interested in what's actually the stride when moving accross the image. And couldn't it be interesting passing it as parameter to the loss?
I had similar problems for "big" kernel sizes.
I could overcome them by using the implementation from voxelmorph: https://github.com/voxelmorph/voxelmorph/blob/dev/voxelmorph/torch/losses.py
So maybe someone could compare to that code.
I would like to add that "overcoming" was too strongly stated. I also get into trouble with the sent Code, however at considerably higher learning rates and kernel sizes.
@Constantin-Jehn This is a great observation, thank you! I hope to look into the difference in the two LNCC implementations when I get a chance.
thanks for the discussion here, it's an issue handling the zero variance... I've included a patch in this PR to fix it https://github.com/Project-MONAI/MONAI/pull/5473.
@ebrahimebrahim I run your scripts and now the large kernels work fine. it's also quite efficient...
Discussed in https://github.com/Project-MONAI/MONAI/discussions/3463