[Feature request] ROI specific exports

[PeerHerholz]

Hi,

I was just wondering if ROI specific functionality/exports might be a nice feature or outside the scope of the project? E.g., let's say I only want to have and compare auditory cortex ROIs.

Cheers, Peer

[satra]

@PeerHerholz - do you mean to separate out the labels into different roi files? i think that part is relatively easy to do with a few lines of nibabel code, so didn't put it in this wrapper.

however if you mean different parcellations, one should be able to use the existing model on more specific datasets to learn different parcels. perhaps @kaczmarj can lend a hand. more generally though we are hoping to transition the codebase to nobrainer 2.0, which should make retraining and transfer learning a lot easier.

[PeerHerholz]

@satra, somewhat both.

do you mean to separate out the labels into different roi files? i think that part is relatively easy to do with a few lines of nibabel code, so didn't put it in this wrapper.

Yeah, that's what I thought, rather straightforward to implement (maybe also a nice fit for niflows?) and outside the scope of the wrapper functionality.

however if you mean different parcellations, one should be able to use the existing model on more specific datasets to learn different parcels. perhaps @kaczmarj can lend a hand. more generally though we are hoping to transition the codebase to nobrainer 2.0, which should make retraining and transfer learning a lot easier.

That sounds great and I would be happy to help out. I'll give it a try with the current model and let you know. Do you have any idea/ideas/insights on possible differences wrt performance/fit between whole brain and e.g. lobe only parcellations?

[kaczmarj]

hi @PeerHerholz - i would be happy to help you with the nibabel code if you'd like.

Do you have any idea/ideas/insights on possible differences wrt performance/fit between whole brain and e.g. lobe only parcellations?

the current model was trained to label 32x32x32 voxel volumes. we predict on whole brains by separating the volume into non-overlapping blocks of 32x32x32. the volumes should have 1mm^3 voxel sizes. to predict on part of a brain (i.e., not whole-brain), one could take part of the whole volume, separate that into 32x32x32 non-overlapping blocks, predict, and put back together the non-overlapping blocks. does that make sense?

using that procedure, the predictions will take less time because there is less data. what do you mean by "fit"? the accuracy of the predictions should be just as good for lobe only parcellations compared to whole-brain parcellations. if you were to use the mean of the variational weights, the predictions should in fact be identical.

[PeerHerholz]

Ahoi hoi @kaczmarj,

thx for following up on this.

i would be happy to help you with the nibabel code if you'd like

Yeah, happy to work with you on that. Any idea where we should place that sort of functionality, in terms of repo, etc.?

the current model was trained to label 32x32x32 voxel volumes. we predict on whole brains by separating the volume into non-overlapping blocks of 32x32x32. the volumes should have 1mm^3 voxel sizes. to predict on part of a brain (i.e., not whole-brain), one could take part of the whole volume, separate that into 32x32x32 non-overlapping blocks, predict, and put back together the non-overlapping blocks. does that make sense?

Check, although I'm not entirely sure on how to implement that. Let's say I only want to parcellate the temporal lobe based on the corresponding learned parcellation. How would my workflow look like? Sorry for being off here.

using that procedure, the predictions will take less time because there is less data. what do you mean by "fit"? the accuracy of the predictions should be just as good for lobe only parcellations compared to whole-brain parcellations. if you were to use the mean of the variational weights, the predictions should in fact be identical.

Oh yeah, you're right.

[kaczmarj]

@PeerHerholz -

Any idea where we should place that sort of functionality, in terms of repo, etc.?

depending on how the roi separation functionality is implemented, this could go in this repo or some other place. i think that separating out the labels into different files would be the best (read easiest) way to go.

i think i went a little overboard in an earlier comment suggesting that we could predict on a subset of the brain in 32x32x32 blocks. all i wanted to convey here is that in theory, you could feed the network any 32x32x32 voxel block from a scan and it will give you labels. so you could feed it blocks from your rois, and it will give you labels for those rois. i think it would be much easier to predict on the whole brain and then extract the labels in your rois.

i realized that the mapping between the model's output (the integer values) and anatomical names might not be clear. i have created a list below of the structures that each label should represent.

if you want to include only certain labels, then you can mask the numpy arrays. let's say you want to extract only caudate and putamen:

# means is the numpy array of the model predictions
mask = ~np.isin(means, [6, 7])
# zero everything except caudate and putamen predictions
means[mask] = 0
variance[mask] = 0
entropy[mask] = 0

is this the behavior you want? or am i misunderstanding?

Regions in predicted label.
Original numbers from FreeSurfer's colormap lookup table in parentheses.

0
  - Unknown (0)
1
  - Left-Cerebral-White-Matter (2)
  - Right-Cerebral-White-Matter (41)
2
  - Left-Lateral-Ventricle (4)
  - Left-Inf-Lat-Vent (5)
  - 3rd-Ventricle (14)
  - 4th-Ventricle (15)
  - Right-Lateral-Ventricle (43)
  - Right-Inf-Lat-Vent (44)
  - 5th-Ventricle (72)
3
  - Left-Cerebellum-White-Matter (7)
  - Right-Cerebellum-White-Matter (46)
4
  - Left-Cerebellum-Cortex (8)
  - Right-Cerebellum-Cortex (47)
5
  - Left-Thalamus-Proper (10)
  - Right-Thalamus-Proper (49)
6
  - Left-Caudate (11)
  - Right-Caudate (50)
7
  - Left-Putamen (12)
  - Right-Putamen (51)
8
  - Left-Pallidum (13)
  - Right-Pallidum (52)
9
  - Brain-Stem (16)
10
  - Left-Hippocampus (17)
  - Right-Hippocampus (53)
11
  - Left-Amygdala (18)
  - Right-Amygdala (54)
12
  - CSF (24)
13
  - Left-Accumbens-area (26)
  - Right-Accumbens-area (58)
14
  - Left-VentralDC (28)
  - Right-VentralDC (60)
15
  - Corpus_Callosum (192)
  - CC_Posterior (251)
  - CC_Mid_Posterior (252)
  - CC_Central (253)
  - CC_Mid_Anterior (254)
  - CC_Anterior (255)
16
  - ctx-lh-bankssts (1001)
  - ctx-rh-bankssts (2001)
17
  - ctx-lh-caudalanteriorcingulate (1002)
  - ctx-rh-caudalanteriorcingulate (2002)
18
  - ctx-lh-caudalmiddlefrontal (1003)
  - ctx-rh-caudalmiddlefrontal (2003)
19
  - ctx-lh-cuneus (1005)
  - ctx-rh-cuneus (2005)
20
  - ctx-lh-entorhinal (1006)
  - ctx-rh-entorhinal (2006)
21
  - ctx-lh-fusiform (1007)
  - ctx-rh-fusiform (2007)
22
  - ctx-lh-inferiorparietal (1008)
  - ctx-rh-inferiorparietal (2008)
23
  - ctx-lh-inferiortemporal (1009)
  - ctx-rh-inferiortemporal (2009)
24
  - ctx-lh-isthmuscingulate (1010)
  - ctx-rh-isthmuscingulate (2010)
25
  - ctx-lh-lateraloccipital (1011)
  - ctx-rh-lateraloccipital (2011)
26
  - ctx-lh-lateralorbitofrontal (1012)
  - ctx-rh-lateralorbitofrontal (2012)
27
  - ctx-lh-lingual (1013)
  - ctx-rh-lingual (2013)
28
  - ctx-lh-medialorbitofrontal (1014)
  - ctx-rh-medialorbitofrontal (2014)
29
  - ctx-lh-middletemporal (1015)
  - ctx-rh-middletemporal (2015)
30
  - ctx-lh-parahippocampal (1016)
  - ctx-rh-parahippocampal (2016)
31
  - ctx-lh-paracentral (1017)
  - ctx-rh-paracentral (2017)
32
  - ctx-lh-parsopercularis (1018)
  - ctx-rh-parsopercularis (2018)
33
  - ctx-lh-parsorbitalis (1019)
  - ctx-rh-parsorbitalis (2019)
34
  - ctx-lh-parstriangularis (1020)
  - ctx-rh-parstriangularis (2020)
35
  - ctx-lh-pericalcarine (1021)
  - ctx-rh-pericalcarine (2021)
36
  - ctx-lh-postcentral (1022)
  - ctx-rh-postcentral (2022)
37
  - ctx-lh-posteriorcingulate (1023)
  - ctx-rh-posteriorcingulate (2023)
38
  - ctx-lh-precentral (1024)
  - ctx-rh-precentral (2024)
39
  - ctx-lh-precuneus (1025)
  - ctx-rh-precuneus (2025)
40
  - ctx-lh-rostralanteriorcingulate (1026)
  - ctx-rh-rostralanteriorcingulate (2026)
41
  - ctx-lh-rostralmiddlefrontal (1027)
  - ctx-rh-rostralmiddlefrontal (2027)
42
  - ctx-lh-superiorfrontal (1028)
  - ctx-rh-superiorfrontal (2028)
43
  - ctx-lh-superiorparietal (1029)
  - ctx-rh-superiorparietal (2029)
44
  - ctx-lh-superiortemporal (1030)
  - ctx-rh-superiortemporal (2030)
45
  - ctx-lh-supramarginal (1031)
  - ctx-rh-supramarginal (2031)
46
  - ctx-lh-frontalpole (1032)
  - ctx-rh-frontalpole (2032)
47
  - ctx-lh-temporalpole (1033)
  - ctx-rh-temporalpole (2033)
48
  - ctx-lh-transversetemporal (1034)
  - ctx-rh-transversetemporal (2034)
49
  - ctx-lh-insula (1035)
  - ctx-rh-insul (2035)

[satra]

@kaczmarj - it may be worthwhile to add a conversion back to freesurfer scheme step at the end of the kwyk inference. this would likely require going label by label and using bwlabeln to get the two clusters. at present what is unclear is how one would get left from right. we could use the nifti info or the fact that the volume is in LIA

it may be worthwhile retraining kwyk with at least one, if not all separated left and right labels, given that we have access to a lot more scans at this point.

[PeerHerholz]

@kaczmarj

is this the behavior you want? or am i misunderstanding?

Yes, exactly. Thx for the clarification.

@kaczmarj & @satra

I think the possible back-to-freesurfer-scheme step would be great and useful. Couldn't the necessary steps be integrated with the discussed functionality to break the parcellation into the respective ROIs (and files) as those could then be assembled again accordingly?

Re retraining the network: I was about to ask if you also have other parcellations that kwyk could be/was trained on (e.g., Destrieux)?