search

spinalcordtoolbox / template_AMU7T

Data repository containing the AMU7T atlas and template
MIT License
0 stars 1 forks source link

sct_extract_metric error: AMU7T/atlas/info_label.txt IDs numbering #7

Open Nilser3 opened 1 year ago

Nilser3 commented 1 year ago

Description:

When the new labels were initially segmented in the AMU7T atlas, The gray matter labels were segmented into sub labels. Wanted to save original PAM50 IDs.

Example: ID 30 in PAM50 is "GM Left ventral-horn", so the 3 sub labels were split:

image

Likewise, in the AMU7T atlas, in addition to dividing the GM labels, they also added labels such as: Canal-central, Anterior fissure and Septum. Finally having 50 labels:

AMU7T/atlas/info_label.txt ```console # ID (compatible with PAM50 ID), name, file 0, WM Left fasciculus-gracilis, AMU7T_label_00_bin.nii.gz 1, WM Right fasciculus-gracilis, AMU7T_label_01_bin.nii.gz 2, WM Left fasciculus-cuneatus, AMU7T_label_02_bin.nii.gz 3, WM Right fasciculus-cuneatus, AMU7T_label_03_bin.nii.gz 4, WM Left lateral-corticospinal-tract, AMU7T_label_04_bin.nii.gz 5, WM Right lateral-corticospinal-tract, AMU7T_label_05_bin.nii.gz 6, WM Left ventral-spinocerebellar-tract, AMU7T_label_06_bin.nii.gz 7, WM Right ventral-spinocerebellar-tract, AMU7T_label_07_bin.nii.gz 8, WM Left rubrospinal-tract, AMU7T_label_08_bin.nii.gz 9, WM Right rubrospinal-tract, AMU7T_label_09_bin.nii.gz 10, WM Left lateral-reticulospinal-tract, AMU7T_label_10_bin.nii.gz 11, WM Right lateral-reticulospinal-tract, AMU7T_label_11_bin.nii.gz 12, WM Left spinal-lemniscus-spinothalamic, AMU7T_label_12_bin.nii.gz 13, WM Right spinal-lemniscus-spinothalamic, AMU7T_label_13_bin.nii.gz 14, WM Left spino-olivary-tract, AMU7T_label_14_bin.nii.gz 15, WM Right spino-olivary-tract, AMU7T_label_15_bin.nii.gz 16, WM Left ventrolateral-reticulospinal-tract, AMU7T_label_16_bin.nii.gz 17, WM Right ventrolateral-reticulospinal-tract, AMU7T_label_17_bin.nii.gz 18, WM Left lateral-vestibulospinal-tract, AMU7T_label_18_bin.nii.gz 19, WM Right lateral-vestibulospinal-tract, AMU7T_label_19_bin.nii.gz 20, WM Left ventral-reticulospinal-tract, AMU7T_label_20_bin.nii.gz 21, WM Right ventral-reticulospinal-tract, AMU7T_label_21_bin.nii.gz 22, WM Left ventral-corticospinal-tract, AMU7T_label_22_bin.nii.gz 23, WM Right ventral-corticospinal-tract, AMU7T_label_23_bin.nii.gz 24, WM Left tectospinal-tract, AMU7T_label_24_bin.nii.gz 25, WM Right tectospinal-tract, AMU7T_label_25_bin.nii.gz 26, WM Left medial-reticulospinal-tract, AMU7T_label_26_bin.nii.gz 27, WM Right medial-reticulospinal-tract, AMU7T_label_27_bin.nii.gz 28, WM Left medial-longitudinal-fasciculus, AMU7T_label_28_bin.nii.gz 29, WM Right medial-longitudinal-fasciculus, AMU7T_label_29_bin.nii.gz 301, GM Left medial-ventral-horn, AMU7T_label_301_bin.nii.gz 302, GM Left central-ventral-horn, AMU7T_label_302_bin.nii.gz 303, GM Left lateral-ventral-horn, AMU7T_label_303_bin.nii.gz 311, GM Right medial-ventral-horn, AMU7T_label_311_bin.nii.gz 312, GM Right central-ventral-horn, AMU7T_label_312_bin.nii.gz 313, GM Right lateral-ventral-horn, AMU7T_label_313_bin.nii.gz 321, GM Left dorsal-intermediate-zone, AMU7T_label_321_bin.nii.gz 322, GM Left ventral-intermediate-zone, AMU7T_label_322_bin.nii.gz 331, GM Right dorsal-intermediate-zone, AMU7T_label_331_bin.nii.gz 332, GM Right ventral-intermediate-zone, AMU7T_label_332_bin.nii.gz 341, GM Left reticular-formation, AMU7T_label_341_bin.nii.gz 342, GM Left dorsomarginal-nucleus, AMU7T_label_342_bin.nii.gz 343, GM Left fasciculus-dorsolateralis, AMU7T_label_343_bin.nii.gz 351, GM Right reticular-formation, AMU7T_label_351_bin.nii.gz 352, GM Right dorsomarginal-nucleus, AMU7T_label_352_bin.nii.gz 353, GM Right fasciculus-dorsolateralis, AMU7T_label_353_bin.nii.gz 36, CSF contour, AMU7T_label_36_bin.nii.gz 37, central-zone, AMU7T_label_37_bin.nii.gz 38, anterior-fissure, AMU7T_label_38_bin.nii.gz 39, septum, AMU7T_label_39_bin.nii.gz # Combined labels # ID, name, IDgroup 30, GM left ventral horn, 301:303 31, GM right ventral horn, 311:313 32, GM left intermediate zone, 321:322 33, GM right intermediate zone, 331:332 34, GM left dorsal horn, 341:343 35, GM right dorsal horn, 351:353 50, spinal cord, 0:353 51, white matter, 0:29 52, gray matter, 301:353 53, dorsal columns, 0:3 54, lateral funiculi, 4:13 55, ventral funiculi, 14:29 ```

However, after registration of the AMU7T template and atlas and trying to use the sct_extract_metric command line, we get this error:

image

Nilser3 commented 1 year ago

After investigating this error, it was observed that it was linked to the IDs of the AMU7T/atlas/info_label.txt, so a new version of AMU7T is proposed in this branch with a consecutive numbering of IDs, with the possibility of merging labels and that works correctly with sct_extract_metric :

sct_extract_metric use: ```console (base) nilser@nilser-NL4x-NL5xLU:~/template_AMU7T/code$ sct_extract_metric -i image.nii.gz -f label_v3/atlas -method wa -o wa_T1.csv -z 2:17 -perslice 1 -- Spinal Cord Toolbox (git-master-61ed4a71ceca1ad43b1ccffd76ca8c24ec90a1ad) sct_extract_metric -i image.nii.gz -f label_v3/atlas -method wa -o wa_T1.csv -z 2:17 -perslice 1 -- Load metric image... Estimation for label: WM_Left_fasciculus-gracilis Estimation for label: WM_Right_fasciculus-gracilis Estimation for label: WM_Left_fasciculus-cuneatus Estimation for label: WM_Right_fasciculus-cuneatus Estimation for label: WM_Left_lateral-corticospinal Estimation for label: WM_Right_lateral-corticospinal Estimation for label: WM_Left_ventral-spinocerebellar Estimation for label: WM_Right_ventral-spinocerebellar Estimation for label: WM_Left_rubrospinal Estimation for label: WM_Right_rubrospinal Estimation for label: WM_Left_lateral-reticulospinal Estimation for label: WM_Right_lateral-reticulospinal Estimation for label: WM_Left_spinal-lemniscus-spinothalamic Estimation for label: WM_Right_spinal-lemniscus-spinothalamic Estimation for label: WM_Left_spino-olivary Estimation for label: WM_Right_spino-olivary Estimation for label: WM_Left_ventrolateral-reticulospinal Estimation for label: WM_Right_ventrolateral-reticulospinal Estimation for label: WM_Left_lateral-vestibulospinal Estimation for label: WM_Right_lateral-vestibulospinal Estimation for label: WM_Left_ventral-reticulospinal Estimation for label: WM_Right_ventral-reticulospinal Estimation for label: WM_Left_ventral-corticospinal Estimation for label: WM_Right_ventral-corticospinal Estimation for label: WM_Left_tectospinal Estimation for label: WM_Right_tectospinal Estimation for label: WM_Left_medial-reticulospinal Estimation for label: WM_Right_medial-reticulospinal Estimation for label: WM_Left_medial-longitudinal-fasciculus Estimation for label: WM_Right_medial-longitudinal-fasciculus Estimation for label: anterior-fissure Estimation for label: septum Estimation for label: GM_Left_medial-ventral-horn Estimation for label: GM_Left_central-ventral-horn Estimation for label: GM_Left_lateral-ventral-horn Estimation for label: GM_Right_medial-ventral-horn Estimation for label: GM_Right_central-ventral-horn Estimation for label: GM_Right_lateral-ventral-horn Estimation for label: GM_Left_dorsal-intermediate-zone Estimation for label: GM_Left_ventral-intermediate-zone Estimation for label: GM_Right_dorsal-intermediate-zone Estimation for label: GM_Right_ventral-intermediate-zone Estimation for label: GM_Left_reticular-formation Estimation for label: GM_Left_dorsomarginal-nucleus Estimation for label: GM_Left_fasciculus-dorsolateralis Estimation for label: GM_Right_reticular-formation Estimation for label: GM_Right_dorsomarginal-nucleus Estimation for label: GM_Right_fasciculus-dorsolateralis Estimation for label: central-canal Estimation for label: CSF-contour Estimation for label: GM_Left_ventral-horn Estimation for label: GM_Right_ventral-horn Estimation for label: GM_Left_intermediate-zone Estimation for label: GM_Right_intermediate-zone Estimation for label: GM_Left_dorsal-horn Estimation for label: GM_Right_dorsal-horn Estimation for label: spinal-cord Estimation for label: white-matter Estimation for label: gray-matter Estimation for label: dorsal-columns Estimation for label: lateral-funiculi Estimation for label: ventral-funiculi Done! To view results, type: xdg-open wa_T1.csv ```

New AMU7T IDs

Finally, the new AMU7T ID nomenclature is compatible with PAM50 only in WM labels (ID 0 - 29).

New AMU7T IDs: ```console # ID, name, file 0, WM_Left_fasciculus-gracilis, AMU7T_label_00_bin.nii.gz 1, WM_Right_fasciculus-gracilis, AMU7T_label_01_bin.nii.gz 2, WM_Left_fasciculus-cuneatus, AMU7T_label_02_bin.nii.gz 3, WM_Right_fasciculus-cuneatus, AMU7T_label_03_bin.nii.gz 4, WM_Left_lateral-corticospinal, AMU7T_label_04_bin.nii.gz 5, WM_Right_lateral-corticospinal, AMU7T_label_05_bin.nii.gz 6, WM_Left_ventral-spinocerebellar, AMU7T_label_06_bin.nii.gz 7, WM_Right_ventral-spinocerebellar, AMU7T_label_07_bin.nii.gz 8, WM_Left_rubrospinal, AMU7T_label_08_bin.nii.gz 9, WM_Right_rubrospinal, AMU7T_label_09_bin.nii.gz 10, WM_Left_lateral-reticulospinal, AMU7T_label_10_bin.nii.gz 11, WM_Right_lateral-reticulospinal, AMU7T_label_11_bin.nii.gz 12, WM_Left_spinal-lemniscus-spinothalamic, AMU7T_label_12_bin.nii.gz 13, WM_Right_spinal-lemniscus-spinothalamic, AMU7T_label_13_bin.nii.gz 14, WM_Left_spino-olivary, AMU7T_label_14_bin.nii.gz 15, WM_Right_spino-olivary, AMU7T_label_15_bin.nii.gz 16, WM_Left_ventrolateral-reticulospinal, AMU7T_label_16_bin.nii.gz 17, WM_Right_ventrolateral-reticulospinal, AMU7T_label_17_bin.nii.gz 18, WM_Left_lateral-vestibulospinal, AMU7T_label_18_bin.nii.gz 19, WM_Right_lateral-vestibulospinal, AMU7T_label_19_bin.nii.gz 20, WM_Left_ventral-reticulospinal, AMU7T_label_20_bin.nii.gz 21, WM_Right_ventral-reticulospinal, AMU7T_label_21_bin.nii.gz 22, WM_Left_ventral-corticospinal, AMU7T_label_22_bin.nii.gz 23, WM_Right_ventral-corticospinal, AMU7T_label_23_bin.nii.gz 24, WM_Left_tectospinal, AMU7T_label_24_bin.nii.gz 25, WM_Right_tectospinal, AMU7T_label_25_bin.nii.gz 26, WM_Left_medial-reticulospinal, AMU7T_label_26_bin.nii.gz 27, WM_Right_medial-reticulospinal, AMU7T_label_27_bin.nii.gz 28, WM_Left_medial-longitudinal-fasciculus, AMU7T_label_28_bin.nii.gz 29, WM_Right_medial-longitudinal-fasciculus, AMU7T_label_29_bin.nii.gz 30, anterior-fissure, AMU7T_label_30_bin.nii.gz 31, septum, AMU7T_label_31_bin.nii.gz 32, GM_Left_medial-ventral-horn, AMU7T_label_32_bin.nii.gz 33, GM_Left_central-ventral-horn, AMU7T_label_33_bin.nii.gz 34, GM_Left_lateral-ventral-horn, AMU7T_label_34_bin.nii.gz 35, GM_Right_medial-ventral-horn, AMU7T_label_35_bin.nii.gz 36, GM_Right_central-ventral-horn, AMU7T_label_36_bin.nii.gz 37, GM_Right_lateral-ventral-horn, AMU7T_label_37_bin.nii.gz 38, GM_Left_dorsal-intermediate-zone, AMU7T_label_38_bin.nii.gz 39, GM_Left_ventral-intermediate-zone, AMU7T_label_39_bin.nii.gz 40, GM_Right_dorsal-intermediate-zone, AMU7T_label_40_bin.nii.gz 41, GM_Right_ventral-intermediate-zone, AMU7T_label_41_bin.nii.gz 42, GM_Left_reticular-formation, AMU7T_label_42_bin.nii.gz 43, GM_Left_dorsomarginal-nucleus, AMU7T_label_43_bin.nii.gz 44, GM_Left_fasciculus-dorsolateralis, AMU7T_label_44_bin.nii.gz 45, GM_Right_reticular-formation, AMU7T_label_45_bin.nii.gz 46, GM_Right_dorsomarginal-nucleus, AMU7T_label_46_bin.nii.gz 47, GM_Right_fasciculus-dorsolateralis, AMU7T_label_47_bin.nii.gz 48, central-canal, AMU7T_label_48_bin.nii.gz 49, CSF-contour, AMU7T_label_49_bin.nii.gz # Combined labels # ID, name, IDgroup 50, GM_Left_ventral-horn, 32:34 51, GM_Right_ventral-horn, 35:37 52, GM_Left_intermediate-zone, 38:39 53, GM_Right_intermediate-zone, 40:41 54, GM_Left_dorsal-horn, 42:44 55, GM_Right_dorsal-horn, 45:47 56, spinal-cord, 0:48 57, white-matter, 0:31 58, gray-matter, 32:48 59, dorsal-columns, 0:3 60, lateral-funiculi, 4:13 61, ventral-funiculi, 14:29 # Clusters used for the first step of the MAP estimation (for advanced users only) # Name, IDgroup white-matter, 0:31 gray-matter, 32:48 CSF, 49 ```
Nilser3 commented 11 months ago

According to what was discussed at the SCT meeting 2023-09-21,

I will use for this experience 3 GM labels and 1 merged label from list of IDs :

The first calculation is carried out by labels 32, 33 and 34

sct_extract_metric -i image.nii.gz -f label/atlas -l 32,33,34 -method wa -o slic_24_label_32_33_34.csv -z 24 -perslice 1

the second calculation in the label 50 (32:34)

sct_extract_metric -i image.nii.gz -f label/atlas -l 50 -method wa -o slic_24_label_50.csv -z 24 -perslice 1

the third calculation in the label 50 (manual merge of 32,33 and 34 labels)

sct_extract_metric -i image.nii.gz -f label/atlas -l 50 -method wa -o slic_24_label_50_merged.csv -z 24 -perslice 1

image

In the right column (lighter coloring), I have tried to reproduce the calculation of averages and standard deviation,

I was able to reproduce the WA calculation, but I had problems with the standard deviation (predict 52,54 and calculated 27,96). I'm checking pixel by pixel because I can't reproduce the STD

Nilser3 commented 11 months ago

Apparently I was misapplying the formula for calculating the standard deviation, To do it correctly, I had to extract all the pixels from the T1q image and the labels (32,33,34 and 50) in slice 24.

image Here is the excel where the calculations were made

Finally I was able to recalculate the WA, size and STD of labels 32,33,34 and 50 independently of sct_extract_metric, and these values are the same as those observed when applying sct_extract_metric both selected in each label, or using the Combined labels approach.

jcohenadad commented 11 months ago

great investigations @Nilser3 ! how about the ML and MAP methods?

Nilser3 commented 11 months ago

It is still complicated to manually calculate the ML and MAP methods, So I started directly using the sct_extract_metric , following the previous experiments,

One Slice

The ML and MAP were calculated in the slice 24 in each IDs 32,33 and 34, also in ID 50 (Combined Labels approach) and in 50 merged (manually agreed masks). Experiments performed on non-binary masks (output of sct_warp_template) and on binarized masks (thr 0.5).

image

I note that the sum of the ML of 32, 33 and 34 is equal to the ML of '50 merged' in non binary and binary cases but I consider that it is complicated to apply and interpret the ML and MAP estimation methods in such a small data set (2 to12 pixels).

Therefore, the methods were applied in 8 slices (19:26)

Eight Slices

image

With all this, I still can't understand why the ML and MAP results between ID 50 and '50 merged' are different (even positive and negative).

I hope your comments will help me understand this problem.

jcohenadad commented 11 months ago

@Nilser3 why not integrating these checks as additional unit tests in sct_extract_metric? also tagging @joshuacwnewton @mguaypaq