Verify and manually correct results for r20200722

[jcohenadad]

With this new release https://github.com/spine-generic/data-multi-subject/releases/tag/r20200722, QC and manual correction needs to be performed and then uploaded to this repository. More specifically:

• Check the cause of the errors (log files with prefix err.)
• Run sg_generate_figure, check for outliers and report the cause for these outliers
• Create a .yml file that lists the data to correct, upload it here, Julien will package the data for correction (with sg_package_for_correction)
• Run manual correction (with sg_manual_correction)

Details for doing the QC available here

[valosekj]

So far, I have checked QC for DWI data using moco_dwi_mean keyword in "Search" feature. I list below my findings.

worse SC segmentation using sct_deepseg_sc in a few slices: sub-vuiisAchieva01_dwi_moco_dwi_mean.nii.gz sub-vuiisAchieva04_dwi_moco_dwi_mean.nii.gz sub-vuiisAchieva06_dwi_moco_dwi_mean.nii.gz sub-perform03_dwi_moco_dwi_mean.nii.gz sub-stanford01_dwi_moco_dwi_mean.nii.gz sub-sapienza01_dwi_moco_dwi_mean.nii.gz sub-mniS03_dwi_moco_dwi_mean.nii.gz

no template warped using sct_warp_template: sub-mountSinai03_dwi_moco_dwi_mean.nii.gz

a few slices of template missing after warping using sct_warp_template: sub-dresden02_dwi_moco_dwi_mean.nii.gz sub-queensland04_dwi_moco_dwi_mean.nii.gz

a few slices of template badly warped using sct_warp_template: sub-stanford01_dwi_moco_dwi_mean.nii.gz sub-sapienza01_dwi_moco_dwi_mean.nii.gz sub-sapienza05_dwi_moco_dwi_mean.nii.gz sub-ubc02_dwi_moco_dwi_mean.nii.gz sub-nottwil01_dwi_moco_dwi_mean.nii.gz sub-beijingVerio02_dwi_concat_moco_dwi_mean.nii.gz sub-oxfordFmrib01_dwi_moco_dwi_mean.nii.gz - totally bad

bad FOV?: sub-beijingPrisma03_dwi_moco_dwi_mean.nii.gz

bad image quality: sub-perform01_dwi_moco_dwi_mean.nii.gz sub-beijingVerio02_dwi_concat_moco_dwi_mean.nii.gz sub-vuiisAchieva04_dwi_moco_dwi_mean.nii.gz

[PaulBautin]

I can not find any vertebrae labeling in QC for t2w images. Is this normal?

[PaulBautin]

Here is the list of QC false segmentation or labelings for t1w and t2w images. I can also send the .yml file:

FILES_SEG:

• sub-beijingGE03_T1w_RPI_r.nii.gz
• sub-brnoUhb02_T1w_RPI_r.nii.gz
• sub-brnoUhb05_T1w_RPI_r.nii.gz
• sub-brnoUhb07_T1w_RPI_r.nii.gz
• sub-brnoUhb08_T1w_RPI_r.nii.gz
• sub-pavia04_T1w_RPI_r.nii.gz
• sub-perform02_T1w_RPI_r.nii.gz
• sub-perform03_T1w_RPI_r.nii.gz
• sub-perform06_T1w_RPI_r.nii.gz
• sub-cmrra02_T2w_RPI_r.nii.gz
• sub-tokyo750w02_T2w_RPI_r.nii.gz
• sub-tokyo750w06_T2w_RPI_r.nii.gz

FILES_GMSEG:

• sub-brnoUhb01_acq-T1w_MTS_crop.nii.gz

FILES_LABEL:

• sub-brnoUhb01_T1w_RPI_r.nii.gz
• sub-brnoUhb08_T1w_RPI_r.nii.gz

[jcohenadad]

@valosekj and @PaulBautin thank you for the review 💯

once https://github.com/neuropoly/spinalcordtoolbox/pull/2734 is merged (should be first thing tomorrow), we can merge https://github.com/spine-generic/spine-generic/pull/146, and then you can start doing the manual correction.

[charleygros]

FILES SCSEG: Some files (example) have an unexpected "rms" prefix.

[jcohenadad]

Some files (example) have an unexpected "rms" prefix.

Some Philips sites included T2star as 4D data (all echoes), so we had to average all echoes. We didn't want to overwrite the data so we decided to add the rms suffix. @charleygros how and where is this suffix unexpected? (e.g. are you working in a specific pipeline that is expecting exact file names based on the spine-generic documentation? are you referring to the BIDS convention? (this is a derivatives), etc.). Thanks

[charleygros]

Some files (example) have an unexpected "rms" prefix.

Some Philips sites included T2star as 4D data (all echoes), so we had to average all echoes. We didn't want to overwrite the data so we decided to add the rms suffix. @charleygros how and where is this suffix unexpected? (e.g. are you working in a specific pipeline that is expecting exact file names based on the spine-generic documentation? are you referring to the BIDS convention? (this is a derivatives), etc.). Thanks

Sorry @jcohenadad ... I expressed myself poorly. I got this issue when using our loader, which look at the image name, eg sub-tokyoIngenia04/anat/sub-tokyoIngenia04_T2star (here), then go in the derivatives/labels folder to grab the associated segmentation with a given suffix, eg _gmseg-manual. So in this example, I would expect to find derivatives/labels/sub-tokyoIngenia04/anat/sub-tokyoIngenia04_T2star_gmseg-manual.nii.gz... which wasn't the case because of this rms suffix! Does it make sense?

A solution would be to upload on the repo the average image, ie sub-tokyoIngenia04/anat/sub-tokyoIngenia04_T2star_rms.nii.gz, so that we could use the GT that you generated on the averaged image.

[jcohenadad]

ahhhhh! of course! i didn't think of that problem 🧐. well that's annoying. I would like to avoid adding processed files under the main BIDS folder (i.e. root/sub-*), as this is against BIDS conventions (i.e. only unprocessed data can go on the root-- the rest is derivatives).

so, I see two solutions: 1) we create a derivatives/sct/ folder under this repos

• problem: people downloading this repos will also download these collateral files (which are specific to a project) 2) we create another repos, dedicated to ivadomed (or other projects), that only includes the derivatives (labels/ sct/) of interest. Will the loader be happy if you point the "raw" data to derivatives/sct?

[jcohenadad]

^ discussion to be continued to another issue https://github.com/spine-generic/data-multi-subject/issues/13

as this issue is specifically about the manual correction of some of the processed datasets

[jcohenadad]

@valosekj can you please make a yml file with the files listed in https://github.com/spine-generic/data-multi-subject/issues/9#issuecomment-662897952 so i can package you the data for the manual corrections?

[jcohenadad]

@PaulBautin can you please upload a yml file for the files listed in https://github.com/spine-generic/data-multi-subject/issues/9#issuecomment-663263703 so i can package the data for you

[PaulBautin]

@jcohenadad, yml file cannot be uploaded in github comments. Should i send it directly to you or in database_mri slack?

[jcohenadad]

yml file cannot be uploaded in github comments.

well, make it a zip 😉

[PaulBautin]

Here is the zipped yml file with the list of QC false segmentations or labelings for t1w and t2w images err_file_t1w_t2w.zip

[jcohenadad]

FileNotFoundError: [Errno 2] No such file or directory: 'results-multi_20200722/data_processed/sub-brnoUhb01/anat/sub-brnoUhb01_acq-T1w_MTS_crop_gmseg.nii.gz'

@PaulBautin you probably made a mistake-- sub-brnoUhb01_acq-T1w_MTS_crop.nii.gz is not used for GM seg

[PaulBautin]

Ah, i'm actually not sure about this one. Could you look at it ==>

[jcohenadad]

you’re looking at warp_template function, not deepseg_gm.

[valosekj]

@valosekj can you please make a yml file with the files listed in #9 (comment) so i can package you the data for the manual corrections?

I am attaching yml file below. Because only SC segmentation is done on DWI data, I included filenames only under FILES_SEG key (I left FILES_GMSEG and FILES_LABEL empty).

I also included a few subjects with low data quality under FILES_SEG key to verify their images in high-quality:

`sub-perform01_dwi_moco_dwi_mean.nii.gz`: `sub-beijingVerio02_dwi_concat_moco_dwi_mean.nii.gz`: `sub-vuiisAchieva04_dwi_moco_dwi_mean.nii.gz`:

How I mentioned in https://github.com/spine-generic/data-multi-subject/issues/9#issuecomment-662897952, there are also subjects with poor template --> dwi registration. In several cases, these subjects are these with bad SC segmentation, so registration of template will be probably improved after manual SC correction. But also, a few subjects with sufficient SC segmentation have then poor template --> dwi registration, e.g.

sub-ubc02_dwi_moco_dwi_mean.nii.gz

Sufficient SC segmentation: Poor template --> dwi registration: 

Have we determined way how to improve this registration for problematic subjects? [err_file_dwi_valosek_29-07-2020.yml.zip](https://github.com/spine-generic/data-multi-subject/files/4993740/err_file_dwi_valosek_29-07-2020.yml.zip)

[PaulBautin]

you’re looking at warp_template function, not deepseg_gm.

Oups my bad, should i remove it from the list or set as a different name of category?

[jcohenadad]

you’re looking at warp_template function, not deepseg_gm.

Oups my bad, should i remove it from the list or set as a different name of category?

please read the doc: https://spine-generic.readthedocs.io/en/latest/analysis_pipeline.html#quality-control

if doc is unclear, pls open issue

[PaulBautin]

Ok, i have to verify the GM segmentation. sorry for the delay.

[PaulBautin]

Here is the corrected zipped yml file with the list of QC false segmentations or labelings for t1w and t2w images. err_file_t1w_t2w_corrected.zip

[jcohenadad]

@valosekj can you please make a yml file with the files listed in #9 (comment) so i can package you the data for the manual corrections?

I am attaching yml file below. Because only SC segmentation is done on DWI data, I included filenames only under FILES_SEG key (I left FILES_GMSEG and FILES_LABEL empty).

I also included a few subjects with low data quality under FILES_SEG key to verify their images in high-quality: How I mentioned in #9 (comment), there are also subjects with poor template --> dwi registration. In several cases, these subjects are these with bad SC segmentation, so registration of template will be probably improved after manual SC correction. But also, a few subjects with sufficient SC segmentation have then poor template --> dwi registration, e.g.

sub-ubc02_dwi_moco_dwi_mean.nii.gz Have we determined way how to improve this registration for problematic subjects?

i'm looking into it

err_file_dwi_valosek_29-07-2020.yml.zip

--> i'll generate a package and @alexfoias will send it to you

@valosekj i've create a specific issue for DWI data quality: https://github.com/spine-generic/data-multi-subject/issues/14 can you please report the low data quality in this issue, with screenshots from your QC report as you did, but without the segmentation overlaid (so we can look at the data).

[jcohenadad]

@valosekj here are the packaged data: data_dwi_valosek_29-07-2020.zip

[jcohenadad]

@PaulBautin @valosekj i've clarified the doc for uploading corrected files: https://spine-generic.readthedocs.io/en/latest/analysis_pipeline.html#upload-the-manually-corrected-files

[PaulBautin]

Should a segmentation that cannot be corrected be discarded? What should be done with original subject (non segmented)? example: sub-cmrra02_T2w_RPI_r.nii.gz:

[jcohenadad]

this is just a problem of QC visu— open the actual seg and image.

[PaulBautin]

Hmm, could it be an error while defacing?

[jcohenadad]

sh... indeed, it is. thank you for spotting this @PaulBautin 🏅

@alexfoias please fix this, thanks.

[jcohenadad]

no template warped using sct_warp_template: sub-mountSinai03_dwi_moco_dwi_mean.nii.gz

@valosekj this is caused by https://github.com/spine-generic/spine-generic/issues/76

so we should indeed exclude this site from the next analysis. I will upload a sct_run_batch.yml in this repository with the sites to exclude.

[alexfoias]

@PaulBautin related to https://github.com/spine-generic/data-multi-subject/issues/9#issuecomment-665953657

Is it sub-cmrra02_T2w ?

[PaulBautin]

yes, sub-cmrra02_T2w_RPI_r.nii.gz

[alexfoias]

@jcohenadad I cannot find on duke/sct_testing/bids the multi-subject dataset before defacing. I also looked on duke/mri, but I cannot find cmrra. Do we have somewhere the original data (I checked in my email but I couldn't find cmrra)?

[jcohenadad]

@alexfoias could you get an old backup from duke? that’s what it’s for

[jcohenadad]

manually-corrected data now released: https://github.com/spine-generic/data-multi-subject/releases/tag/r20200801

great job @valosekj and @PaulBautin 💯