How do I get the stat files?

sanderslab / magellanmapper

MagellanMapper is a graphical interface for 3D bioimage annotation, atlas registration, and regional quantification

BSD 3-Clause "New" or "Revised" License

21 stars 3 forks source link

How do I get the stat files? #51

Closed SaibotMagd closed 2 years ago

SaibotMagd commented 3 years ago

Hi,

I can't find a way to create the stat files for the label-eroding & smoothing step on my data. I'm looking for the files created in the basic protocol (I don't know which step but I only did everything till protocol 5)

ADMBA-E18pt5_smoothing.csv
ADMBA-E18pt5_smoothing_raw.csv
ADMBA-E18pt5_stats.csv

When I rerun the protocol 5 they won't be recreate.

I only get the markers.csv. So how can I get them?

yoda-vid commented 3 years ago

Hi @SaibotMagd, great question! Those files are output to the atlas folder when running atlas import with smoothing on, but they're not currently generating when running the watershed-based smoothing (--register merge_atlas_segs). I think this was just to save time, but I can definitely add the output for the smoothing metrics as well. I'll add a profile switch too so it's optional.

Thanks for pointing this out!

yoda-vid commented 3 years ago

I've added a branch/PR that outputs smoothing stats now during the eroding/smoothing step. Let me know if it works for you!

SaibotMagd commented 3 years ago

I've added a branch/PR that outputs smoothing stats now during the eroding/smoothing step. Let me know if it works for you!

Thanks again! I think its best to print the stats as default. Its working but I only get: "xx_smoothing.csv & xx_smoothing_raw.csv" but not the summary: "xx_stats.csv".

yoda-vid commented 3 years ago

Thanks for testing this! I've updated the PR to output the _stats.csv file as well. It includes the volume and overlap metrics but skips the lateral planes metrics since this step doesn't involve the lateral extension. Let me know how it works for you.

SaibotMagd commented 3 years ago

Mistake on my terms. I'm sorry.

yoda-vid commented 3 years ago

Thanks for testing and the stack trace. Hm, haven't seen this one before and curious that it works in one orientation but not the other. Are you running this on 2D planes? Do you have any sample files I could try to reproduce this error?

I pushed an update that fixes another issue with LabelOverlapMeasuresImageFilter (#52). It looks different than the issue here but could possibly have fixed it too.

SaibotMagd commented 3 years ago

I think the mistake was on my side. It seems like it is impossible to do the "make_edge_images" step using images from different directions after another without deleting the files created before.

Now it's working. Now i have to try to understand the meaning of the statistics. So #45 is now becoming more and more pressing. Because it seems like the coronal segmentation is to good. I don't think it makes sense to refine it because I don't see enough room for improvement. So all comes to the horizontal refinement step. What if I could get the DSC statistics for the coronal slices without doing the refinement to compare it with the refinement result from the horizontal view? Could it been used as a measure of quality between the point of views?

Pre-refined horizontal: Pre-refined corontal:

yoda-vid commented 3 years ago

I think the mistake was on my side. It seems like it is impossible to do the "make_edge_images" step using images from different directions after another without deleting the files created before.

Sounds good, no problem. Yes, the make_edge_images step does need to be rerun before running the merge_atlas_segs step if the original images are changed. Is this what you were encountering? make_edge_images shouldn't require manually deleting files ahead of time, but if so let me know.

Because it seems like the coronal segmentation is to good. I don't think it makes sense to refine it because I don't see enough room for improvement. So all comes to the horizontal refinement step. What if I could get the DSC statistics for the coronal slices without doing the refinement to compare it with the refinement result from the horizontal view? Could it been used as a measure of quality between the point of views?

I agree that the coronal segmentations look quite smooth as-is. I may still be a bit confused about your exact setup, trying to understand why the horizontal vs coronal refinements give different outputs since the refinement (and DSC) works in 3D and should give the same output whether the input is given coronally or horizontally (though I have mainly tested atlases given in one orientation). Or are you doing successive 2D plane refinements on either 2D coronal or 2D horizontal planes rather than inputting a single 3D volume?

The DSC could be a way to compare the atlas outputs. The DSC output saved in the file just compares the foreground of the intensity image and aggregate labels, while the command-line output also shows the DSC of the original to refined labels to show how much the refinement has changed the labels (though not necessarily the quality of those changes). If you want to measure the quality of the refinement compared with histological boundaries, you could use the --register vol_stats tasks for metrics such as the intensity variation within each label to see if variation decreases with refinement (each label is more homogeneous) and distances to the histological edges.

./run.py <path-to-img> --register vol_stats --labels <path-to-labels>

yoda-vid commented 3 years ago

I posted a PR (#57) that fixes the atlas-to-labels DSC to use the new rather than the original labels image and also saves the label-to-labels DSC rather than just printing them to console.

SaibotMagd commented 3 years ago

The DSC could be a way to compare the atlas outputs. The DSC output saved in the file just compares the foreground of the intensity image and aggregate labels, while the command-line output also shows the DSC of the original to refined labels to show how much the refinement has changed the labels (though not necessarily the quality of those changes). If you want to measure the quality of the refinement compared with histological boundaries, you could use the --register vol_stats tasks for metrics such as the intensity variation within each label to see if variation decreases with refinement (each label is more homogeneous) and distances to the histological edges.
./run.py <path-to-img> --register vol_stats --labels <path-to-labels>

This seems not to work for me:

magmap.io.cli - INFO - Launched MagellanMapper from /home/toolboxes/magellanmapper/run.py
root - INFO - Set filenames to ['/home/toolboxes/magellanmapper/examples/fullres/fullres_atlasVolume.nii'], current filename /home/toolboxes/magellanmapper/examples/fullres/fullres_atlasVolume.nii
root - INFO - Set default (channel 0) ROI profile: default
root - INFO - Set atlas profile to default
root - INFO - Set labels to {<AtlasLabels.PATH_REF: 1>: '/home/toolboxes/magellanmapper/examples/fullres/fullres_annotation.nii', <AtlasLabels.LEVEL: 2>: None, <AtlasLabels.ID: 3>: None, <AtlasLabels.ORIG_COLORS: 4>: 1, <AtlasLabels.SYMMETRIC_COLORS: 5>: True, <AtlasLabels.BINARY: 6>: None, <AtlasLabels.TRANSLATE_LABELS: 7>: None, <AtlasLabels.TRANSLATE_CHILDREN: 8>: None}
root - INFO - Set register type to vol_stats
root - INFO - setting up Matplotlib style default
root - INFO - applying theme DEFAULT
root - INFO - Loaded database from /home/.local/share/MagellanMapper/magmap.db
root - INFO - set multiprocessing start method to fork
root - INFO - Performing register task: RegisterTypes.VOL_STATS
magmap.atlas.ontology - ERROR - Error tokenizing data. C error: Expected 1 fields in line 320762, saw 6136
Traceback (most recent call last):
  File "/home/toolboxes/magellanmapper/magmap/atlas/ontology.py", line 53, in load_labels_ref
    labels_ref = pd.read_csv(path)
  File "/home/anaconda3/envs/mag/lib/python3.6/site-packages/pandas/io/parsers.py", line 688, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "/home/anaconda3/envs/mag/lib/python3.6/site-packages/pandas/io/parsers.py", line 460, in _read
    data = parser.read(nrows)
  File "/home/anaconda3/envs/mag/lib/python3.6/site-packages/pandas/io/parsers.py", line 1198, in read
    ret = self._engine.read(nrows)
  File "/home/anaconda3/envs/mag/lib/python3.6/site-packages/pandas/io/parsers.py", line 2157, in read
    data = self._reader.read(nrows)
  File "pandas/_libs/parsers.pyx", line 847, in pandas._libs.parsers.TextReader.read
  File "pandas/_libs/parsers.pyx", line 862, in pandas._libs.parsers.TextReader._read_low_memory
  File "pandas/_libs/parsers.pyx", line 918, in pandas._libs.parsers.TextReader._read_rows
  File "pandas/_libs/parsers.pyx", line 905, in pandas._libs.parsers.TextReader._tokenize_rows
  File "pandas/_libs/parsers.pyx", line 2042, in pandas._libs.parsers.raise_parser_error
pandas.errors.ParserError: Error tokenizing data. C error: Expected 1 fields in line 320762, saw 6136

root - CRITICAL - Unhandled exception. Additional log saved to: /tmp/magellanmapper_error_xyb2skpe.log
Traceback (most recent call last):
  File "/home/toolboxes/magellanmapper/magmap/atlas/ontology.py", line 53, in load_labels_ref
    labels_ref = pd.read_csv(path)
  File "/home/anaconda3/envs/mag/lib/python3.6/site-packages/pandas/io/parsers.py", line 688, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "/home/anaconda3/envs/mag/lib/python3.6/site-packages/pandas/io/parsers.py", line 460, in _read
    data = parser.read(nrows)
  File "/home/anaconda3/envs/mag/lib/python3.6/site-packages/pandas/io/parsers.py", line 1198, in read
    ret = self._engine.read(nrows)
  File "/home/anaconda3/envs/mag/lib/python3.6/site-packages/pandas/io/parsers.py", line 2157, in read
    data = self._reader.read(nrows)
  File "pandas/_libs/parsers.pyx", line 847, in pandas._libs.parsers.TextReader.read
  File "pandas/_libs/parsers.pyx", line 862, in pandas._libs.parsers.TextReader._read_low_memory
  File "pandas/_libs/parsers.pyx", line 918, in pandas._libs.parsers.TextReader._read_rows
  File "pandas/_libs/parsers.pyx", line 905, in pandas._libs.parsers.TextReader._tokenize_rows
  File "pandas/_libs/parsers.pyx", line 2042, in pandas._libs.parsers.raise_parser_error
pandas.errors.ParserError: Error tokenizing data. C error: Expected 1 fields in line 320762, saw 6136

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/toolboxes/magellanmapper/run.py", line 237, in <module>
    main()
  File "/home/toolboxes/magellanmapper/run.py", line 170, in main
    launch_magmap()
  File "/home/toolboxes/magellanmapper/run.py", line 126, in launch_magmap
    cli.main()
  File "/home/toolboxes/magellanmapper/magmap/io/cli.py", line 872, in main
    process_tasks()
  File "/home/toolboxes/magellanmapper/magmap/io/cli.py", line 724, in process_tasks
    register.main()
  File "/home/toolboxes/magellanmapper/magmap/atlas/register.py", line 1996, in main
    extra_metrics=extra_metric_groups)
  File "/home/toolboxes/magellanmapper/magmap/atlas/register.py", line 1322, in volumes_by_id
    ontology.load_labels_ref(labels_ref_path))
  File "/home/toolboxes/magellanmapper/magmap/atlas/ontology.py", line 57, in load_labels_ref
    f"Could not load labels reference file from '{path}', skipping")
FileNotFoundError: Could not load labels reference file from '/home/toolboxes/magellanmapper/examples/fullres/fullres_annotation.nii', skipping
Starting MagellanMapper run script...
Out[7]: 256

I'm curious: What's the difference between the "--img $folder" parameter and the imagecall without the "--img" flag: "run.py atlasVolume.nii"?

SaibotMagd commented 3 years ago

Update: What do you mean by: "--labels "? I tried it with the example data using "ontology17.json" as the labels file and it worked. So do I need such a json file?

yoda-vid commented 3 years ago

I'm curious: What's the difference between the "--img $folder" parameter and the imagecall without the "--img" flag: "run.py atlasVolume.nii"?

I made --img optional in v1.4 as a convenience when the image path is the first argument since it's used with most commands. Keeping --img still works and also lets you put it elsewhere in the command (eg ./run.py --register vol_stats --img <path-to-img> --labels <path-to-labels>, where the image path isn't the first argument).

What do you mean by: "--labels "? I tried it with the example data using "ontology17.json" as the labels file and it worked. So do I need such a json file?

Glad it works now! The --labels parameter is for the labels reference/ontology file, eg the ontology17.json file that you used, which is needed to get the labels and their hierarchy. The labels/annotation image file path doesn't need to be specified here since it's found automatically based on the main image name, assuming it's been registered to the main image.

I just posted a PR (#61) so that you can use a flat CSV file like the region_ids.csv file in the sample data instead of a JSON file. The CSV can even have just a single column, Regions, with the IDs of each of the labels to measure. Let me know how it works for you.

I've also been working on a document for all the command line flags to summarize them and hopefully make things easier. It's a work in progress so let me know if you have any suggestions for it (and feel free to edit/PR)!

https://github.com/sanderslab/magellanmapper/blob/master/docs/cli.md

SaibotMagd commented 3 years ago

Glad it works now! The --labels parameter is for the labels reference/ontology file, eg the ontology17.json file that you used, which is needed to get the labels and their hierarchy. The labels/annotation image file path doesn't need to be specified here since it's found automatically based on the main image name, assuming it's been registered to the main image.

What does it mean "it's been registered"? Yes I registered the annotations to my ls-image, but I never did this with magellan only external.

I just posted a PR (#61) so that you can use a flat CSV file like the region_ids.csv file in the sample data instead of a JSON file. The CSV can even have just a single column, Regions, with the IDs of each of the labels to measure. Let me know how it works for you.

I tried this but it isn't working, at least for a 2D image. I create a csv file only consisting of the regions in this particular image (np.unique). I also tried to copy the other columns from the regions_ids.csv, so it looks like: grafik

This is the error log:

magmap.io.cli - INFO - Launched MagellanMapper from /home/toolboxes/magellanmapper/run.py
root - INFO - Set filenames to ['/home/toolboxes/magellanmapper/examples/singleSlice'], current filename /home/toolboxes/magellanmapper/examples/singleSlice
root - INFO - Set default (channel 0) ROI profile: default
root - INFO - Set atlas profile to default
root - INFO - Set labels to {<AtlasLabels.PATH_REF: 1>: '/home/toolboxes/magellanmapper/examples/singleSlice/region_ids.csv', <AtlasLabels.LEVEL: 2>: None, <AtlasLabels.ID: 3>: None, <AtlasLabels.ORIG_COLORS: 4>: 1, <AtlasLabels.SYMMETRIC_COLORS: 5>: True, <AtlasLabels.BINARY: 6>: None, <AtlasLabels.TRANSLATE_LABELS: 7>: None, <AtlasLabels.TRANSLATE_CHILDREN: 8>: None}
root - INFO - Set register type to vol_stats
root - INFO - setting up Matplotlib style default
root - INFO - applying theme DEFAULT
root - INFO - Loaded database from /home/.local/share/MagellanMapper/magmap.db
root - INFO - set multiprocessing start method to fork
root - INFO - Performing register task: RegisterTypes.VOL_STATS
root - CRITICAL - Unhandled exception. Additional log saved to: /tmp/magellanmapper_error_cskzah_z.log
Traceback (most recent call last):
  File "/home/anaconda3/envs/mag/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 2898, in get_loc
    return self._engine.get_loc(casted_key)
  File "pandas/_libs/index.pyx", line 70, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 101, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 1675, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 1683, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'msg'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/toolboxes/magellanmapper/run.py", line 237, in <module>
    main()
  File "/home/toolboxes/magellanmapper/run.py", line 170, in main
    launch_magmap()
  File "/home/toolboxes/magellanmapper/run.py", line 126, in launch_magmap
    cli.main()
  File "/home/toolboxes/magellanmapper/magmap/io/cli.py", line 872, in main
    process_tasks()
  File "/home/toolboxes/magellanmapper/magmap/io/cli.py", line 724, in process_tasks
    register.main()
  File "/home/toolboxes/magellanmapper/magmap/atlas/register.py", line 1996, in main
    extra_metrics=extra_metric_groups)
  File "/home/toolboxes/magellanmapper/magmap/atlas/register.py", line 1322, in volumes_by_id
    ontology.load_labels_ref(labels_ref_path))
  File "/home/toolboxes/magellanmapper/magmap/atlas/ontology.py", line 122, in create_aba_reverse_lookup
    labels_ref["msg"][0], config.ABAKeys.ABA_ID.value,
  File "/home/anaconda3/envs/mag/lib/python3.6/site-packages/pandas/core/frame.py", line 2906, in __getitem__
    indexer = self.columns.get_loc(key)
  File "/home/anaconda3/envs/mag/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 2900, in get_loc
    raise KeyError(key) from err
KeyError: 'msg'
Starting MagellanMapper run script...
Out[27]: 256

yoda-vid commented 3 years ago

What does it mean "it's been registered"? Yes I registered the annotations to my ls-image, but I never did this with magellan only external.

The registration outputs paths to specific filenames, eg _annotation.mhd or .nii.gz at the end of labels images, and vol_stats just looks for those paths.

Is your folder structure like this?

/home/toolboxes/magellanmapper/examples/singleSlice:
atlasVolume.nii
annotation.nii

This should also work since I made a special case for files in the ABA convention. You can give the folder path as the img argument, and it should find the associated labels image file. The log output should also show where it's looking for files in case it doesn't find it, and you can move it there.

I tried this but it isn't working, at least for a 2D image. I create a csv file only consisting of the regions in this particular image (np.unique). I also tried to copy the other columns from the regions_ids.csv, so it looks like

The CSV file looks good. Volume stats with the CSV version doesn't follow label hierarchies yet, so something I'll need to add. The error looks like what I got before. Did you try it on the PR's branch (access_df_lookup)? I haven't merged it in yet, so it's still just in that branch.

SaibotMagd commented 3 years ago

The registration outputs paths to specific filenames, eg _annotation.mhd or .nii.gz at the end of labels images, and vol_stats just looks for those paths.

I'm sorry, I don't understand what that means.

Is your folder structure like this?

/home/toolboxes/magellanmapper/examples/singleSlice:
atlasVolume.nii
annotation.nii

Yes that's how it is. + region_ids.csv , just to be sure: the 3 columns on the right are "fakes", I just copied them from ADMBA-E18pt5/region_ids.csv and added my grey-values in the first column.

This should also work since I made a special case for files in the ABA convention. You can give the folder path as the img argument, and it should find the associated labels image file. The log output should also show where it's looking for files in case it doesn't find it, and you can move it there.

Yes the error message mention something about the ABA atlas template. But what does this have to do with the calculation of the intensity values? Could it be that he try to find an ID for a particular grey-value in the ABA atlas, because my annotation-file doesn't share all of the grey-values from the ABA atlas because it isn't a regular mouse ;). So probably he can not find a label for a grey-value?

The CSV file looks good. Volume stats with the CSV version doesn't follow label hierarchies yet, so something I'll need to add. The error looks like what I got before. Did you try it on the PR's branch (access_df_lookup)? I haven't merged it in yet, so it's still just in that branch.

I'm totally new with all the git branch, pull request stuff, so I missed that. But now I tried it also using the new branch.

So the error-log changed a little bit:

magmap.io.cli - INFO - Launched MagellanMapper from /home/toolboxes/magellanmapper/run.py
root - INFO - Set filenames to ['/home/toolboxes/magellanmapper/examples/singleSlice/'], current filename /home/toolboxes/magellanmapper/examples/singleSlice/
root - INFO - Set default (channel 0) ROI profile: default
root - INFO - Set atlas profile to default
root - INFO - Set labels to {<AtlasLabels.PATH_REF: 1>: '/home/toolboxes/magellanmapper/examples/singleSlice/region_ids.csv', <AtlasLabels.LEVEL: 2>: None, <AtlasLabels.ID: 3>: None, <AtlasLabels.ORIG_COLORS: 4>: 1, <AtlasLabels.SYMMETRIC_COLORS: 5>: True, <AtlasLabels.BINARY: 6>: None, <AtlasLabels.TRANSLATE_LABELS: 7>: None, <AtlasLabels.TRANSLATE_CHILDREN: 8>: None}
root - INFO - Set register type to vol_stats
root - INFO - setting up Matplotlib style default
root - INFO - applying theme DEFAULT
root - INFO - Loaded database from /home/.local/share/MagellanMapper/magmap.db
root - INFO - set multiprocessing start method to fork
root - INFO - Performing register task: RegisterTypes.VOL_STATS
root - CRITICAL - Unhandled exception. Additional log saved to: /tmp/magellanmapper_error_n44hbm0n.log
Traceback (most recent call last):
  File "/home/anaconda3/envs/mag/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 2898, in get_loc
    return self._engine.get_loc(casted_key)
  File "pandas/_libs/index.pyx", line 70, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 101, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 1675, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 1683, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'msg'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/toolboxes/magellanmapper/run.py", line 237, in <module>
    main()
  File "/home/toolboxes/magellanmapper/run.py", line 170, in main
    launch_magmap()
  File "/home/toolboxes/magellanmapper/run.py", line 126, in launch_magmap
    cli.main()
  File "/home/toolboxes/magellanmapper/magmap/io/cli.py", line 872, in main
    process_tasks()
  File "/home/toolboxes/magellanmapper/magmap/io/cli.py", line 724, in process_tasks
    register.main()
  File "/home/toolboxes/magellanmapper/magmap/atlas/register.py", line 1996, in main
    extra_metrics=extra_metric_groups)
  File "/home/toolboxes/magellanmapper/magmap/atlas/register.py", line 1322, in volumes_by_id
    ontology.load_labels_ref(labels_ref_path))
  File "/home/toolboxes/magellanmapper/magmap/atlas/ontology.py", line 122, in create_aba_reverse_lookup
    labels_ref["msg"][0], config.ABAKeys.ABA_ID.value,
  File "/home/anaconda3/envs/mag/lib/python3.6/site-packages/pandas/core/frame.py", line 2906, in __getitem__
    indexer = self.columns.get_loc(key)
  File "/home/anaconda3/envs/mag/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 2900, in get_loc
    raise KeyError(key) from err
KeyError: 'msg'
Starting MagellanMapper run script...
Out[53]: 256

yoda-vid commented 3 years ago

I'm totally new with all the git branch, pull request stuff, so I missed that. But now I tried it also using the new branch.

No worries! Thanks for trying the new branch. The error still looks like the old one though. The error is showing that the file wasn't loaded because it's treating it as a JSON file, which the new branch updates to detect CSV files. If you do git log, does it show the commit, fa9aa37dfedfcfa2c66f2ce404a7f041920164c9 , from PR #61? Does git fetch; git checkout access_df_lookup bring in the new code?

But what does this have to do with the calculation of the intensity values? Could it be that he try to find an ID for a particular grey-value in the ABA atlas, because my atlasVolume doesn't share all of the grey-values from the ABA atlas because it isn't a regular mouse ;).

By "grey-values," do you mean the label integer values in your annotations image? The region IDs column is used to list the label integer-defined regions to extract from the annotations/labels image (not the intensity image). Is that what you were referring to?

I'm sorry, I don't understand what that means.

The "registered images" expects an atlas folder structure like this:

"Base" path: /home/me/my_image
Intensity/histology/"atlas" image: /home/me/my_image_atlasVolume.nii.gz
Annotation/labels image: /home/me/my_image_annotation.nii.gz
Other registered images, eg atlas edges: /home/me/my_image_atlasEdge.nii.gz
Load intensity and annotations image: ./run.py --img /home/me/my_image --reg_suffixes atlasVolume.nii.gz annotation.nii.gz

So the base path can be given as the --img argument, and all the registered images are found based on it, without having to give their full paths. The intention was to make things shorter and more convenient, though I realize it can be confusing (always open to suggestions!). I recently added support to give an absolute path for the registered images (eg --reg_suffixes annotation=/home/me/my_image_atlasEdge.nii.gz) in case it has a different base path. Does this clarify anything you referred to in #26?

Your folder structure is slightly different, where the base path is a folder, and atlasVolume.nii.gz and annotation.nii.gz are full filenames, similar to the ABA file structure. This is treated as a special case and should also find the related images.

SaibotMagd commented 3 years ago

Sorry for my late answer, I'm facing some very serious issues regarding other parts of my pipeline.

But what does this have to do with the calculation of the intensity values? Could it be that he try to find an ID for a particular grey-value in the ABA atlas, because my atlasVolume doesn't share all of the grey-values from the ABA atlas because it isn't a regular mouse ;).

By "grey-values," do you mean the label integer values in your annotations image? The region IDs column is used to list the label integer-defined regions to extract from the annotations/labels image (not the intensity image). Is that what you were referring to?

Refering to "atlasVolume" in my post before was a mistake. It should be "annotations". So every grey-value in the annotation file refer to one ID from the csv-label file. And there're 100% identical in each region. On the other hand in the "atlasVolume" File the grey-values in a particular region aren't homogenious and do not match the IDs from the label-file. So it was a spelling mistake.

yoda-vid commented 3 years ago

Sounds good. I've merged in PR #61 so that it's on the main branch now. I've also opened #67 (now merged) with additional updates to volume stats such as merging in the region names automatically and support for hierarchical labels loaded from CSV files, if that's relevant.

Feel free to suggest any other stats that might be worth measuring and integrating to optimize your results. Hope this helps.

SaibotMagd commented 3 years ago

After doing "--register vol_stats" on the ABA_25um_atlas I got this output:


        original    L         57        NaN           NaN         NaN           NaN            NaN           NaN           NaN            NaN               NaN       NaN           NaN           NaN            NaN
        original    L         65        NaN           NaN         NaN           NaN            NaN           NaN           NaN            NaN               NaN       NaN           NaN           NaN            NaN
        original    L        624        NaN           NaN         NaN           NaN            NaN           NaN           NaN            NaN               NaN       NaN           NaN           NaN            NaN
        original    L  304325711        NaN           NaN         NaN           NaN            NaN           NaN           NaN            NaN               NaN       NaN           NaN           NaN            NaN
root - INFO - Sample Condition  Side Region      Volume  Nuclei  Density  RegVolMean  RegNucMean  RegDensityMean  VarNuclei  VarIntensity   EdgeSize   EdgeDistSum  EdgeDistMean
        original  both    all  462.409828     NaN      NaN    4.585572         NaN             NaN        NaN   5059.815171  7060879.0  2.179814e+13     83.573098
root - INFO - time elapsed to measure variation: 57.54461669921875
root - INFO - Backed up /home/tgottsch/toolboxes/magellanmapper/examples/allen25um/_volumes.csv to /home/tgottsch/toolboxes/magellanmapper/examples/allen25um/_volumes(1).csv
root - INFO - exported volume data per sample to CSV file: "/home/tgottsch/toolboxes/magellanmapper/examples/allen25um/_volumes.csv"
root - INFO - Backed up vols_by_sample.csv to vols_by_sample(2).csv
root - INFO - exported volume data per sample to CSV file: "vols_by_sample.csv"
root - INFO - Backed up vols_by_sample_summary.csv to vols_by_sample_summary(2).csv
root - INFO - exported volume data per sample to CSV file: "vols_by_sample_summary.csv"
root - INFO - time elapsed for volumes by ID: 59.92438530921936
Starting MagellanMapper run script...
Out[39]: 0

"_volumes.csv" exists, but neither "vols_by_sample.csv" nor "vols_by_sample_summary.csv" can be found.

SaibotMagd commented 3 years ago

I tried to run '--vol_stats' on the E11pt5-Dataset from https://search.kg.ebrains.eu/instances/Project/b8a8e2d3-4787-45f2-b010-589948c33f20 and got this error:

magmap.io.cli - INFO - Launched MagellanMapper from /home/saibotMagd/toolboxes/magellanmapper/run.py
root - INFO - Set filenames to ['/home/saibotMagd/srcImageData/metricCalc/ADMBA-E11pt5_pub/3Drecon-ADMBA-E11pt5'], current filename /home/saibotMagd/srcImageData/metricCalc/ADMBA-E11pt5_pub/3Drecon-ADMBA-E11pt5
root - INFO - Set default (channel 0) ROI profile: default
root - INFO - Set atlas profile to default
root - INFO - Set labels to {<AtlasLabels.PATH_REF: 1>: '/home/saibotMagd/srcImageData/metricCalc/ADMBA-E11pt5_pub/3Drecon-ADMBA-E11pt5region_ids_ADMBA.csv', <AtlasLabels.LEVEL: 2>: None, <AtlasLabels.ID: 3>: None, <AtlasLabels.ORIG_COLORS: 4>: 1, <AtlasLabels.SYMMETRIC_COLORS: 5>: True, <AtlasLabels.BINARY: 6>: None, <AtlasLabels.TRANSLATE_LABELS: 7>: None, <AtlasLabels.TRANSLATE_CHILDREN: 8>: None}
root - INFO - Set register type to vol_stats
root - INFO - setting up Matplotlib style default
root - INFO - applying theme DEFAULT
root - INFO - Loaded database from /home/saibotMagd/.local/share/MagellanMapper/magmap.db
root - INFO - set multiprocessing start method to fork
root - INFO - Performing register task: RegisterTypes.VOL_STATS
root - CRITICAL - Unhandled exception. Additional log saved to: /tmp/magellanmapper_error_cyubic_o.log
Traceback (most recent call last):
  File "/home/saibotMagd/anaconda3/envs/mag/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 2898, in get_loc
    return self._engine.get_loc(casted_key)
  File "pandas/_libs/index.pyx", line 70, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 101, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 1675, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 1683, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'msg'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/saibotMagd/toolboxes/magellanmapper/run.py", line 237, in <module>
    main()
  File "/home/saibotMagd/toolboxes/magellanmapper/run.py", line 170, in main
    launch_magmap()
  File "/home/saibotMagd/toolboxes/magellanmapper/run.py", line 126, in launch_magmap
    cli.main()
  File "/home/saibotMagd/toolboxes/magellanmapper/magmap/io/cli.py", line 872, in main
    process_tasks()
  File "/home/saibotMagd/toolboxes/magellanmapper/magmap/io/cli.py", line 724, in process_tasks
    register.main()
  File "/home/saibotMagd/toolboxes/magellanmapper/magmap/atlas/register.py", line 1996, in main
    extra_metrics=extra_metric_groups)
  File "/home/saibotMagd/toolboxes/magellanmapper/magmap/atlas/register.py", line 1322, in volumes_by_id
    ontology.load_labels_ref(labels_ref_path))
  File "/home/saibotMagd/toolboxes/magellanmapper/magmap/atlas/ontology.py", line 122, in create_aba_reverse_lookup
    labels_ref["msg"][0], config.ABAKeys.ABA_ID.value,
  File "/home/saibotMagd/anaconda3/envs/mag/lib/python3.6/site-packages/pandas/core/frame.py", line 2906, in __getitem__
    indexer = self.columns.get_loc(key)
  File "/home/saibotMagd/anaconda3/envs/mag/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 2900, in get_loc
    raise KeyError(key) from err
KeyError: 'msg'
Starting MagellanMapper run script...
Out[132]: 256

yoda-vid commented 3 years ago

I tried to run '--vol_stats' on the E11pt5-Dataset from https://search.kg.ebrains.eu/instances/Project/b8a8e2d3-4787-45f2-b010-589948c33f20 and got this error:

magmap.io.cli - INFO - Launched MagellanMapper from /home/saibotMagd/toolboxes/magellanmapper/run.py
root - INFO - Set filenames to ['/home/saibotMagd/srcImageData/metricCalc/ADMBA-E11pt5_pub/3Drecon-ADMBA-E11pt5'], current filename /home/saibotMagd/srcImageData/metricCalc/ADMBA-E11pt5_pub/3Drecon-ADMBA-E11pt5
root - INFO - Set default (channel 0) ROI profile: default
root - INFO - Set atlas profile to default
root - INFO - Set labels to {<AtlasLabels.PATH_REF: 1>: '/home/saibotMagd/srcImageData/metricCalc/ADMBA-E11pt5_pub/3Drecon-ADMBA-E11pt5region_ids_ADMBA.csv', <AtlasLabels.LEVEL: 2>: None, <AtlasLabels.ID: 3>: None, <AtlasLabels.ORIG_COLORS: 4>: 1, <AtlasLabels.SYMMETRIC_COLORS: 5>: True, <AtlasLabels.BINARY: 6>: None, <AtlasLabels.TRANSLATE_LABELS: 7>: None, <AtlasLabels.TRANSLATE_CHILDREN: 8>: None}

I think this needs another slash in 3Drecon-ADMBA-E11pt5region_ids_ADMBA.csv so that it's 3Drecon-ADMBA-E11pt5/region_ids_ADMBA.csv? It's working otherwise for me but let me know if you still get an error.

"_volumes.csv" exists, but neither "vols_by_sample.csv" nor "vols_by_sample_summary.csv" can be found.

The "vols_by_sample.csv" and "vols_by_sample_summary.csv" files actually get output to the current working directory, whereas the "_volumes.csv" file is output to the atlas directory. "vols_by_sample.csv" and "_volumes.csv" are the same here, but "vols_by_sample.csv" includes all atlases when running vol_stats on multiple atlases at the same time. Do you see those files in the working directory?

Thanks and hope that helps!

SaibotMagd commented 3 years ago

I found the "vol_by_sample.csv" and "vols_bz_sample_summary.csv" in my home directory, so I think I messed up the "find file command". But either way the content of these files don't help at all. I still don't know the command to calculate the DSC and intensity metrics. So I called the "meas_dice" function right from atlas_stats.py on every label mask by hand, but the result is poor. Also tried to calculate the intensity values and weight them using the label-annotation sizes. But the results are to bad to use them.

I renamed the region_id file again but it isn't working:

bash = run + \
        ' --img ' + '/home/saibotMagd/srcImageData/metricCalc/ADMBA-E11pt5_pub/3Drecon-ADMBA-E11pt5' + \
        " --register vol_stats " + \
        ' --labels ' + '/home/saibotMagd/srcImageData/metricCalc/ADMBA-E11pt5_pub/region_ids_ADMBA.csv'
os.system(bash)
magmap.io.cli - INFO - Launched MagellanMapper from /home/saibotMagd/toolboxes/magellanmapper/run.py
root - INFO - Set filenames to ['/home/saibotMagd/srcImageData/metricCalc/ADMBA-E11pt5_pub/3Drecon-ADMBA-E11pt5'], current filename /home/saibotMagd/srcImageData/metricCalc/ADMBA-E11pt5_pub/3Drecon-ADMBA-E11pt5
root - INFO - Set default (channel 0) ROI profile: default
root - INFO - Set atlas profile to default
root - INFO - Set labels to {<AtlasLabels.PATH_REF: 1>: '/home/saibotMagd/srcImageData/metricCalc/ADMBA-E11pt5_pub/region_ids_ADMBA.csv', <AtlasLabels.LEVEL: 2>: None, <AtlasLabels.ID: 3>: None, <AtlasLabels.ORIG_COLORS: 4>: 1, <AtlasLabels.SYMMETRIC_COLORS: 5>: True, <AtlasLabels.BINARY: 6>: None, <AtlasLabels.TRANSLATE_LABELS: 7>: None, <AtlasLabels.TRANSLATE_CHILDREN: 8>: None}
root - INFO - Set register type to vol_stats
root - INFO - setting up Matplotlib style default
root - INFO - applying theme DEFAULT
root - INFO - Loaded database from /home/saibotMagd/.local/share/MagellanMapper/magmap.db
root - INFO - set multiprocessing start method to fork
root - INFO - Performing register task: RegisterTypes.VOL_STATS
root - CRITICAL - Unhandled exception. Additional log saved to: /tmp/magellanmapper_error_yjvj9_1m.log
Traceback (most recent call last):
  File "/home/saibotMagd/anaconda3/envs/mag/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 2898, in get_loc
    return self._engine.get_loc(casted_key)
  File "pandas/_libs/index.pyx", line 70, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 101, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 1675, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 1683, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'msg'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/saibotMagd/toolboxes/magellanmapper/run.py", line 237, in <module>
    main()
  File "/home/saibotMagd/toolboxes/magellanmapper/run.py", line 170, in main
    launch_magmap()
  File "/home/saibotMagd/toolboxes/magellanmapper/run.py", line 126, in launch_magmap
    cli.main()
  File "/home/saibotMagd/toolboxes/magellanmapper/magmap/io/cli.py", line 872, in main
    process_tasks()
  File "/home/saibotMagd/toolboxes/magellanmapper/magmap/io/cli.py", line 724, in process_tasks
    register.main()
  File "/home/saibotMagd/toolboxes/magellanmapper/magmap/atlas/register.py", line 1996, in main
    extra_metrics=extra_metric_groups)
  File "/home/saibotMagd/toolboxes/magellanmapper/magmap/atlas/register.py", line 1322, in volumes_by_id
    ontology.load_labels_ref(labels_ref_path))
  File "/home/saibotMagd/toolboxes/magellanmapper/magmap/atlas/ontology.py", line 122, in create_aba_reverse_lookup
    labels_ref["msg"][0], config.ABAKeys.ABA_ID.value,
  File "/home/saibotMagd/anaconda3/envs/mag/lib/python3.6/site-packages/pandas/core/frame.py", line 2906, in __getitem__
    indexer = self.columns.get_loc(key)
  File "/home/saibotMagd/anaconda3/envs/mag/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 2900, in get_loc
    raise KeyError(key) from err
KeyError: 'msg'
Starting MagellanMapper run script...

yoda-vid commented 3 years ago

I renamed the region_id file again but it isn't working

This reminds of the error fixed in #61, now updated on the main branch, so maybe that will fix it?

But the results are to bad to use them.

I'm sorry that the stats aren't working out for you. Perhaps we can discuss this more to see what results you're getting and how to troubleshoot.

SaibotMagd commented 2 years ago

I did it! To make this possible I had to run the whole pipeline, and throw away the results I don't need:

run make_edges
overwrite the annotationEdge.mhd/raw with my gradient Edge detection result
run merge_atlas
optional: delete or overwrite the annotation.mhd/raw if you don't need the smoothing results
run vol_stats using --labels region_ids.csv
optional: find the vols_by_sample.csv and vols_by_sample_summary.csv and move them to the working directory (for me they always end up in the system directory not in the working)

This is the result and it seems great: as you can see the magellanmapper results (left) are identical to the statistics from itk-snap (right).

I thought VarIntensity would be the variance but it stands for "variation".

yoda-vid commented 2 years ago

Excellent! Glad it worked out, and thanks for reporting your steps on how to make this work. I'll definitely need to add this into the docs.

Thanks for all your help with this!