Closed gdevenyi closed 9 years ago
ntustison
commented
9 years ago You'll have to explore this issue a bit on your end before we investigate since it's not clear that it's an ITK issue. For example, the files are called "transformed". Are we dealing with a float interpolated image? Can you reproduce with other file formats aside from mnc?
gdevenyi
commented
9 years ago The mnc file is a set of labels that were transformed from one space to another rigidly with the following command:
mincresample -keep_real_range -unsigned -byte -clobber -near -transform ../../OASIS_0061_t1-OASIS_2061_t1.xfm -like ../../../ImageSet2/labels/OASIS_2061_t1_labels.mnc ../../../ImageSet1/labels/OASIS_0061_t1_labels.mnc OASIS_0061_t1_labels_transformed.mncThis is a test-retest of running the same automated segmentation algorithm on two T1 scans of the same subject separated by a couple days, so the goal is to compare the segmentations produced on each volume, register the volumes together (using mincANTS) and use that transformation to transform the labels, then calculate overlap scores.
Properties of the file
> mincinfo OASIS_0061_t1_labels_transformed.mnc
file: OASIS_0061_t1_labels_transformed.mnc
image: unsigned byte 0 to 255
image dimensions: xspace zspace yspace
dimension name length step start
-------------- ------ ---- -----
xspace 170 -1 133
zspace 171 1 1
yspace 232 1 -125I have uploaded the original labels (pre transformation), which show the exact same problem: https://www.dropbox.com/s/44wr706mxyxo5tw/OASIS_0061_t1_labels.mnc?dl=0
I have not tested in another file format because minc is the file format all the other tools in this toolchain are built on.
ntustison
commented
9 years ago Okay, let me be more specific. I don't have much to go on but what it looks like is that the mincresample command is doing some type of interpolation of the transformed labels and I'm guessing LabelOverlapMeasures is truncating the real values whereas volume_similarity is taking the ceiling. It's just a guess but it would explain what's happening. Regardless, I don't think it's a problem with the underlying ITK/ANTs code.
ntustison
commented
9 years ago For example, look here
http://manpages.ubuntu.com/manpages/hardy/man1/mincresample.1.html
It doesn't look like you're using nearest neighbor interpolation.
gdevenyi
commented
9 years ago I have highlighted the nearest neighbour argument in my command:
mincresample -keep_real_range -unsigned -byte -clobber |||||-near||||| -transform ../../OASIS_0061_t1-OASIS_2061_t1.xfm -like ../../../ImageSet2/labels/OASIS_2061_t1_labels.mnc ../../../ImageSet1/labels/OASIS_0061_t1_labels.mnc OASIS_0061_t1_labels_transformed.mncI have now tested this on a nii file through the following pathway MINC2 -> MINC1
mincconvert OASIS_0061_t1_labels_transformed.mnc OASIS_0061_t1_labels_transformed-ver1.mncfreesurfer's mri_convert
mri_convert OASIS_0061_t1_labels_transformed-ver1.mnc OASIS_0061_t1_labels_transformed-ver1.nii> LabelOverlapMeasures 3 OASIS_0061_t1_labels_transformed-ver1.nii OASIS_0061_t1_labels_transformed-ver1.nii
************ All Labels *************
Total Union (jaccard) Mean (dice) Volume sim. False negative False positive
1 1 1 0 0 0
************ Individual Labels *************
Label Target Union (jaccard) Mean (dice) Volume sim. False negative False positive
1 1 1 1 0 0 0
2 1 1 1 0 0 0
5 1 1 1 0 0 0
6 1 1 1 0 0 0
7 1 1 1 0 0 0
13 1 1 1 0 0 0
14 1 1 1 0 0 0
25 1 1 1 0 0 0
38 1 1 1 0 0 0
40 1 1 1 0 0 0
43 1 1 1 0 0 0
116 1 1 1 0 0 0
117 1 1 1 0 0 0
119 1 1 1 0 0 0
121 1 1 1 0 0 0
122 1 1 1 0 0 0
128 1 1 1 0 0 0
255 1 1 1 0 0 0
This set of labels is correct. This confirms for me that it's an issue in the MINC IO component.
gdevenyi
commented
9 years ago So.. is this a "wontfix"?
ntustison
commented
9 years ago No, if it's an issue in the MINC IO, then it's an ITK problem.
stnava
commented
9 years ago right - could be ITK minc i/o or as @ntustison mentioned earlier possibly something related to how types are represented in mnc vs whatever we cast to when we read an image into LabelOverlapMeasures ( probably not char ). you might wan to try a different type on your end and see if that helps.
brian
On Wed, Mar 25, 2015 at 4:12 PM, Nick Tustison notifications@github.com wrote:
No, if it's an issue in the MINC IO, then it's an ITK problem.
— Reply to this email directly or view it on GitHub https://github.com/stnava/ANTs/issues/174#issuecomment-86199954.
ntustison
commented
9 years ago We read the label images as unsigned int
https://github.com/stnava/ANTs/blob/master/Examples/LabelOverlapMeasures.cxx#L40-L41
ntustison
commented
9 years ago Also, I know that the MNC IO in ITK is one of the least mature in the toolkit. In fact, I don't know if it is officially supported yet.
cookpa
commented
9 years ago Hi Gabriel,
Do you still see the problem after making the images integers? ie,
LabelOverlapMeasures 3 OASIS_0061_t1_labels_transformed-ver1.mnc OASIS_0061_t1_labels_transformed-ver1.mnc
?
If the file in your drop box is float, it could be the cause of the problem. There are several labels in the OASIS data set that are not consistently present in the set of atlases. Some of these do not show up consistently in MALF segmentations, or they are small, sometimes one voxel. These labels were ignored in the original segmentation challenge:
ignore_labels = [1:3, 5:10, 12:22, 24:29, 33:34, 42:43, 53:54, 63:68, 70, 74, 80:99, 110:111, 126:127, 130:131, 158:159, 188:189];
https://masi.vuse.vanderbilt.edu/workshop2012/index.php/Challenge_Details
Some of the problem labels are on this list, so they are likely to be very small regions. That plus float interpolation could cause discrepancies after transformation.
On Mar 25, 2015, at 3:02 PM, Gabriel A. Devenyi notifications@github.com wrote:
mincresample -keep_real_range -unsigned -byte -clobber -near
I have now tested this on a nii file through the following pathway MINC2 -> MINC1
mincconvert OASIS_0061_t1_labels_transformed.mnc OASIS_0061_t1_labels_transformed-ver1.mnc
freesurfer's mri_convert
mri_convert OASIS_0061_t1_labels_transformed-ver1.mnc OASIS_0061_t1_labels_transformed-ver1.nii
LabelOverlapMeasures 3 OASIS_0061_t1_labels_transformed-ver1.nii OASIS_0061_t1_labels_transformed-ver1.nii \ All Labels ***** Total Union (jaccard) Mean (dice) Volume sim. False negative False positive 1 1 1 0 0 0 ** Individual Labels ***** Label Target Union (jaccard) Mean (dice) Volume sim. False negative False positive 1 1 1 1 0 0 0 2 1 1 1 0 0 0 5 1 1 1 0 0 0 6 1 1 1 0 0 0 7 1 1 1 0 0 0 13 1 1 1 0 0 0 14 1 1 1 0 0 0 25 1 1 1 0 0 0 38 1 1 1 0 0 0 40 1 1 1 0 0 0 43 1 1 1 0 0 0 116 1 1 1 0 0 0 117 1 1 1 0 0 0 119 1 1 1 0 0 0 121 1 1 1 0 0 0 122 1 1 1 0 0 0 128 1 1 1 0 0 0 255 1 1 1 0 0 0
This set of labels is correct. This confirms for me that it's an issue in the MINC IO component.
gdevenyi
commented
9 years ago @cookpa OASIS_0061_t1_labels_transformed-ver1.mnc is a MINC1 version file (NETCDF), which MINCIO in ITK doesn't support reading at all.
If I
mincresample -unsigned -int OASIS_0061_t1_labels_transformed.mnc OASIS_0061_t1_labels_transformed-int.mncSo that the labels are explicitly stored as unsigned ints the same problem remains.
As I noted above, these are all label minc files, they are stored as unsigned byte in their default format.
With regards to the labels themselves, these are labels generated by our automated segmentation tool MAGeTBrain https://github.com/CobraLab/MAGeTbrain, they are the white/gray matter projections of the hippocampus (fornix, fimbira etc)
stnava
commented
9 years ago ok - this does sound more like an itk problem. is it the case that the untransformed image also has this problem or no?
if you want to bring to attention of itk developers, a small reproducible example of the problem would be helpful.
brian
On Wed, Mar 25, 2015 at 4:56 PM, Gabriel A. Devenyi < notifications@github.com> wrote:
@cookpa https://github.com/cookpa OASIS_0061_t1_labels_transformed-ver1.mnc is a MINC1 version file (NETCDF), which MINCIO in ITK doesn't support reading at all.
If I
mincresample -unsigned -int OASIS_0061_t1_labels_transformed.mnc OASIS_0061_t1_labels_transformed-int.mnc
So that the labels are explicitly stored as unsigned ints the same problem remains.
As I noted above, these are all label minc files, they are stored as unsigned byte in their default format.
With regards to the labels themselves, these are labels generated by our automated segmentation tool MAGeTBrain https://github.com/CobraLab/MAGeTbrain, they are the white/gray matter projections of the hippocampus (fornix, fimbira etc)
— Reply to this email directly or view it on GitHub https://github.com/stnava/ANTs/issues/174#issuecomment-86213118.
gdevenyi
commented
9 years ago The untransformed image shows the same problem, as noted in the third comment above. The file is also linked on dropbox for reference.
stnava
commented
9 years ago ok then - recommend filing ITK bug report.
brian
On Wed, Mar 25, 2015 at 5:07 PM, Gabriel A. Devenyi < notifications@github.com> wrote:
The untransformed image shows the same problem, as noted in the third comment above. The file is also linked on dropbox for reference.
— Reply to this email directly or view it on GitHub https://github.com/stnava/ANTs/issues/174#issuecomment-86218381.
gdevenyi
commented
9 years ago Calling @vfonov MINCIO in ITK seems to be having issues with MINC label files, any thoughts?
vfonov
commented
9 years ago mincresample -unsigned -int XXX will mess up integer labels
MINCIO reads and writes integer labels properly
MINC1 (NETCDF) is supported if you link ITK with externally compiled libminc that has MINC1 part.
vfonov
commented
9 years ago if you have a minc file , use mincheader to inspect the contents of image-min and image-max variables. If you see bunch of different numbers it means that minc file has slice normalization and the integer labels probably got messed up.
gdevenyi
commented
9 years ago Alright, here's a recent test data dump:
mincinfo OAS1_0304_MR1_mpr_n4_anon_sbj_111.n4correct.cutneckapplyautocrop_labels.mnc
file: OAS1_0304_MR1_mpr_n4_anon_sbj_111.n4correct.cutneckapplyautocrop_labels.mnc
image: unsigned byte 0 to 255
image dimensions: zspace yspace xspace
dimension name length step start
-------------- ------ ---- -----
zspace 183 1 -39
yspace 219 1 -116
xspace 168 1 -36mincheader OAS1_0304_MR1_mpr_n4_anon_sbj_111.n4correct.cutneckapplyautocrop_labels.mnc
hdf5 OAS1_0304_MR1_mpr_n4_anon_sbj_111.n4correct.cutneckapplyautocrop_labels {
dimensions:
xspace = 168 ;
yspace = 219 ;
zspace = 183 ;
variables:
byte image(zspace, yspace, xspace) ;
image:dimorder = "zspace,yspace,xspace" ;
image:valid_range = 0., 255. ;
image:signtype = "unsigned" ;
double image-min ;
double image-max ;
double xspace ;
xspace:length = 168 ;
xspace:spacing = "regular__" ;
xspace:class = "spatial" ;
xspace:direction_cosines = 1., 0., 0. ;
xspace:step = 1. ;
xspace:start = -36. ;
xspace:units = "mm" ;
xspace:width = 0. ;
double yspace ;
yspace:length = 219 ;
yspace:spacing = "regular__" ;
yspace:class = "spatial" ;
yspace:direction_cosines = 0., 1., 0. ;
yspace:step = 1. ;
yspace:start = -116. ;
yspace:units = "mm" ;
yspace:width = 0. ;
double zspace ;
zspace:length = 183 ;
zspace:spacing = "regular__" ;
zspace:class = "spatial" ;
zspace:direction_cosines = 0., 0., 1. ;
zspace:step = 1. ;
zspace:start = -39. ;
zspace:units = "mm" ;
zspace:width = 0. ;
// global attributes:
:ident = "gdevenyi:gpc-f124n077-ib0:2015.05.02.10.38.41:8666:1" ;
:minc_version = "2.1.01" ;
:class = "real___" ;
:history = "Sat May 2 10:38:41 2015>>> mincconvert -2 -clob -compress 9 /scratch/m/mchakrav/gdevenyi/OASIS/matt_merged/output/fusion/majority_vote/OAS1_0304_MR1_mpr_n4_anon_sbj_111.n4correct.cutneckapplyautocrop_labels.mnc.v1 /scratch/m/mchakrav/gdevenyi/OASIS/matt_merged/output/fusion/majority_vote/OAS1_0304_MR1_mpr_n4_anon_sbj_111.n4correct.cutneckapplyautocrop_labels.mnc\n",
"" ;
data:
image-min = 0 ;
image-max = 152 ;
xspace = 0 ;
yspace = 0 ;
zspace = 0 ;
}print_all_labels OAS1_0304_MR1_mpr_n4_anon_sbj_111.n4correct.cutneckapplyautocrop_labels.mnc
Label: 1 134
Label: 2 458
Label: 3 1515
Label: 4 2965
Label: 5 6034
Label: 6 11592
Label: 7 5213
Label: 8 2779
Label: 9 4288
Label: 10 2844
Label: 11 2649
Label: 12 460
Label: 13 6955
Label: 14 637
Label: 15 1422
Label: 16 1364
Label: 17 1209
Label: 18 202
Label: 19 560
Label: 20 877
Label: 21 593
Label: 22 948
Label: 23 306
Label: 31 828
Label: 32 321
Label: 33 452
Label: 34 119
Label: 35 460
Label: 36 438
Label: 37 94
Label: 38 560
Label: 39 46
Label: 40 952
Label: 50 8447
Label: 51 1301
Label: 52 4772
Label: 60 113
Label: 80 105461
Label: 102 527
Label: 103 1313
Label: 104 3274
Label: 105 5548
Label: 106 11453
Label: 107 6556
Label: 108 3465
Label: 109 3059
Label: 110 2835
Label: 111 2669
Label: 112 523
Label: 113 6877
Label: 131 865
Label: 132 283
Label: 133 452
Label: 134 133
Label: 135 432
Label: 136 405
Label: 137 78
Label: 138 553
Label: 139 47
Label: 140 931
Label: 150 8046
Label: 151 1279
Label: 152 4513volume_similarity OAS1_0304_MR1_mpr_n4_anon_sbj_111.n4correct.cutneckapplyautocrop_labels.mnc OAS1_0304_MR1_mpr_n4_anon_sbj_111.n4correct.cutneckapplyautocrop_labels.mnc --csv
1,1,1,1,1
2,1,1,1,1
3,1,1,1,1
4,1,1,1,1
5,1,1,1,1
6,1,1,1,1
7,1,1,1,1
8,1,1,1,1
9,1,1,1,1
10,1,1,1,1
11,1,1,1,1
12,1,1,1,1
13,1,1,1,1
14,1,1,1,1
15,1,1,1,1
16,1,1,1,1
17,1,1,1,1
18,1,1,1,1
19,1,1,1,1
20,1,1,1,1
21,1,1,1,1
22,1,1,1,1
23,1,1,1,1
31,1,1,1,1
32,1,1,1,1
33,1,1,1,1
34,1,1,1,1
35,1,1,1,1
36,1,1,1,1
37,1,1,1,1
38,1,1,1,1
39,1,1,1,1
40,1,1,1,1
50,1,1,1,1
51,1,1,1,1
52,1,1,1,1
60,1,1,1,1
80,1,1,1,1
102,1,1,1,1
103,1,1,1,1
104,1,1,1,1
105,1,1,1,1
106,1,1,1,1
107,1,1,1,1
108,1,1,1,1
109,1,1,1,1
110,1,1,1,1
111,1,1,1,1
112,1,1,1,1
113,1,1,1,1
131,1,1,1,1
132,1,1,1,1
133,1,1,1,1
134,1,1,1,1
135,1,1,1,1
136,1,1,1,1
137,1,1,1,1
138,1,1,1,1
139,1,1,1,1
140,1,1,1,1
150,1,1,1,1
151,1,1,1,1
152,1,1,1,1LabelOverlapMeasures 3 OAS1_0304_MR1_mpr_n4_anon_sbj_111.n4correct.cutneckapplyautocrop_labels.mnc OAS1_0304_MR1_mpr_n4_anon_sbj_111.n4correct.cutneckapplyautocrop_labels.mnc
************ All Labels *************
Total Union (jaccard) Mean (dice) Volume sim. False negative False positive
1 1 1 0 0 0
************ Individual Labels *************
Label Target Union (jaccard) Mean (dice) Volume sim. False negative False positive
1 1 1 1 0 0 0
2 1 1 1 0 0 0
4 1 1 1 0 0 0
5 1 1 1 0 0 0
7 1 1 1 0 0 0
8 1 1 1 0 0 0
10 1 1 1 0 0 0
11 1 1 1 0 0 0
13 1 1 1 0 0 0
14 1 1 1 0 0 0
16 1 1 1 0 0 0
17 1 1 1 0 0 0
19 1 1 1 0 0 0
20 1 1 1 0 0 0
22 1 1 1 0 0 0
23 1 1 1 0 0 0
30 1 1 1 0 0 0
32 1 1 1 0 0 0
33 1 1 1 0 0 0
35 1 1 1 0 0 0
36 1 1 1 0 0 0
38 1 1 1 0 0 0
39 1 1 1 0 0 0
50 1 1 1 0 0 0
51 1 1 1 0 0 0
60 1 1 1 0 0 0
79 1 1 1 0 0 0
101 1 1 1 0 0 0
103 1 1 1 0 0 0
104 1 1 1 0 0 0
106 1 1 1 0 0 0
107 1 1 1 0 0 0
109 1 1 1 0 0 0
110 1 1 1 0 0 0
112 1 1 1 0 0 0
113 1 1 1 0 0 0
131 1 1 1 0 0 0
132 1 1 1 0 0 0
134 1 1 1 0 0 0
135 1 1 1 0 0 0
137 1 1 1 0 0 0
138 1 1 1 0 0 0
140 1 1 1 0 0 0
150 1 1 1 0 0 0
152 1 1 1 0 0 0So LabelOverlapMeasures doesn't see all the labels 45 vs 63
vfonov
commented
9 years ago you have mage:valid_range = 0., 255. and image-range 0 152, which means that libminc does whole volume normalization. I.e value of each voxel is divided by 152 and multiplied by 255 before writing it to disk, and vice-versa when reading.
gdevenyi
commented
9 years ago Okay so on the minc side I've always just treated this as a label file and it works fine as is, how do I fix this discrepancy? On Jul 10, 2015 6:13 PM, "Vladimir S. FONOV" notifications@github.com wrote:
you have mage:valid_range = 0., 255. and image-range 0 152, which means that minc does whole volume normalization. I.e for each voxel it devides value by 152 and multiplies by 255 before writing it to disk, and vice-versa when reading.
— Reply to this email directly or view it on GitHub https://github.com/stnava/ANTs/issues/174#issuecomment-120540264.
vfonov
commented
9 years ago
LabelOverlapMeasures is missing some labels when processing minc files:
In comparison, the same file analyzed by minc-toolkit's volume_similarity:
ANTs misses labels 2, 22, 35, 102 and incorrectly detects labels 21, 34, 100, and 103
Reference file available here https://www.dropbox.com/s/5y9rzrdf46tnap7/OASIS_0061_t1_labels_transformed.mnc?dl=0