multi-channel input bug

YipengHu / label-reg

(This repo is no longer up-to-date. Any updates will be at https://github.com/DeepRegNet/DeepReg/) A demo of the re-factored label-driven registration code, based on "Weakly-supervised convolutional neural networks for multimodal image registration"

Apache License 2.0

117 stars 32 forks source link

multi-channel input bug #15

Closed YipengHu closed 5 years ago

YipengHu commented 5 years ago

The provided inference function fails to correctly warp multi-channel inputs (as required for label images). We have written a bug-fix for this, using pytorch's grid_sample function and would be happy to provide this to the interested community.

mattiaspaul commented 5 years ago

thanks for opening this issue, I have a related question, regarding label-reg's treatment of multi-channel labels: from my understanding of the following line in training.py: https://github.com/YipengHu/label-reg/blob/3fe086f8bb9d029acf9fdecba3cee92009cfb3dc/training.py#L75 it appears that for each image-pair in a mini-batch only a single label is chosen in each iteration. While this may be good when dealing with very few and spatially separated structures, I am curious whether it wouldn't be better to train with more (preferably) all labels jointly to avoid opposing gradients in subsequent iterations?

YipengHu commented 5 years ago

This is an insightful comment. Training with multiple labels together requires a multi-class-like loss function, usually needs fixed number of classes pre-defined and (a bit) more memory. What do you think a good one would be? Opposing gradients is an potential issue, one that we actually encountered before. This went away in our case where a very heavily weighted smoothness penalty term was used.

YipengHu commented 5 years ago

I'm closing this as it is replicating the same thing addressed in dev branch.