bglocker
commented
4 years ago I think your issue is due to a confusion about what we mean with masks and segs in the code. The masks are just a way to provide image masks for the intensity normalization. They are not used otherwise and are optional. Masks would be used, for example, for MRI brain scans to mask out the brain when doing the zero-mean-unit-variance normalization.
Structures for the registration, whether it's landmarks or actual segments, will have to be passed via the --train_seg, --val_seg and --test_seg arguments. For point-like structures, I would recommend to create a smoothed version of the landmark maps (similar to the example in the paper). You may want to try the "explicit" loss function. The "supervised" one is a baseline using image intensities only, it doesn't use the provided structures.
I am interested to use the ISTN to try aligning brain scans, leveraging the location of mitochondria as landmarks. However I have run into issues in the code. In my understanding from the ISTN paper tied to this github project, segmentations are landmarks which are more tree/segment shaped and masks are landmarks which are more point shaped. In the paper, both are demonstrated to have good uses with ISTNs for alignment of images and promoting the ITN's ability to make good representations of the input. I am hoping to leverage a configuration that best optimizes for the mitochondria as landmarks, which leads me to want to use a mask; especially if the ISTN can be better optimized for point like structures with masks. The mitochondria tend to have a very point-like presense in the brain tissue as tubular structures. Thus I am more inclined to try using them as a mask rather than like segments. However in trying to pass the mitochondria as masks and leaving the segmentation as None in the istn-reg.py commandline arguments, I run into many issues in the code throughout that seem to require a lot more than just a hotfix or two. Thus I am wondering is there is a way to run the code with masks instead of segmentation? Also which loss function would you think is the best bet for getting good results with mitochondria as masks would you say? Some preliminary results have surprisingly shown the supervised loss function is not doing so well compared to the others.