search

spine-generic / data-multi-subject

Multi-subject data for the Spine Generic project
Creative Commons Attribution 4.0 International
22 stars 15 forks source link

Fix defaced images #119

Closed alexfoias closed 2 years ago

alexfoias commented 2 years ago

Fixes https://github.com/spine-generic/data-multi-subject/issues/115

alexfoias commented 2 years ago

For the moment I have pushed T1w images for the data that we received with face. @renelabounek Do you mind taking a look to QC the defacing ? thanks.

renelabounek commented 2 years ago

Sure, @mbondy023 and I can look at the images within this week. How can I get the pictures on our server? Three tweaks that I do not know how to solve to get the images:

1/ On the server, we have downloaded this fork of the database: https://github.com/renelabounek/data-multi-subject (which is up-to-date with the master database) How can I switch to af/fix_defaced_images branch from this fork?

2/ When I do, do I need to call some git-annex update to get the updated images on the server?

3/ Do you have a list of updated files? Or should we check all files listed in #115?

alexfoias commented 2 years ago

@renelabounek You would have to clone this repo.

After that you have to do:

git checkout af/fix_defaced_images
git annex get .

Then look at the branch to see if it looks right to you.

To investigate what changed:

git log --stat master..HEAD # to see filenames
git log -p master..HEAD     # to see content, commit-by-commit
git diff master..HEAD       # to see content, overall

The updated files are:

``` sub-amu01/anat/sub-amu01_T1w.nii.gz sub-amu02/anat/sub-amu02_T1w.nii.gz sub-amu03/anat/sub-amu03_T1w.nii.gz sub-amu04/anat/sub-amu04_T1w.nii.gz sub-amu05/anat/sub-amu05_T1w.nii.gz sub-balgrist01/anat/sub-balgrist01_T1w.nii.gz sub-balgrist02/anat/sub-balgrist02_T1w.nii.gz sub-balgrist03/anat/sub-balgrist03_T1w.nii.gz sub-balgrist04/anat/sub-balgrist04_T1w.nii.gz sub-balgrist05/anat/sub-balgrist05_T1w.nii.gz sub-balgrist06/anat/sub-balgrist06_T1w.nii.gz sub-brnoCeitec01/anat/sub-brnoCeitec01_T1w.nii.gz sub-brnoCeitec02/anat/sub-brnoCeitec02_T1w.nii.gz sub-brnoCeitec03/anat/sub-brnoCeitec03_T1w.nii.gz sub-brnoCeitec04/anat/sub-brnoCeitec04_T1w.nii.gz sub-brnoCeitec05/anat/sub-brnoCeitec05_T1w.nii.gz sub-brnoCeitec06/anat/sub-brnoCeitec06_T1w.nii.gz sub-brnoUhb01/anat/sub-brnoUhb01_T1w.nii.gz sub-brnoUhb02/anat/sub-brnoUhb02_T1w.nii.gz sub-brnoUhb03/anat/sub-brnoUhb03_T1w.nii.gz sub-brnoUhb04/anat/sub-brnoUhb04_T1w.nii.gz sub-brnoUhb05/anat/sub-brnoUhb05_T1w.nii.gz sub-brnoUhb06/anat/sub-brnoUhb06_T1w.nii.gz sub-brnoUhb07/anat/sub-brnoUhb07_T1w.nii.gz sub-brnoUhb08/anat/sub-brnoUhb08_T1w.nii.gz sub-dresden01/anat/sub-dresden01_T1w.nii.gz sub-dresden02/anat/sub-dresden02_T1w.nii.gz sub-geneva01/anat/sub-geneva01_T1w.nii.gz sub-geneva02/anat/sub-geneva02_T1w.nii.gz sub-geneva03/anat/sub-geneva03_T1w.nii.gz sub-geneva04/anat/sub-geneva04_T1w.nii.gz sub-geneva05/anat/sub-geneva05_T1w.nii.gz sub-geneva06/anat/sub-geneva06_T1w.nii.gz sub-juntendo750w01/anat/sub-juntendo750w01_T1w.nii.gz sub-juntendo750w02/anat/sub-juntendo750w02_T1w.nii.gz sub-juntendo750w03/anat/sub-juntendo750w03_T1w.nii.gz sub-juntendo750w04/anat/sub-juntendo750w04_T1w.nii.gz sub-juntendo750w05/anat/sub-juntendo750w05_T1w.nii.gz sub-juntendo750w06/anat/sub-juntendo750w06_T1w.nii.gz sub-nottwil01/anat/sub-nottwil01_T1w.nii.gz sub-nottwil02/anat/sub-nottwil02_T1w.nii.gz sub-nottwil03/anat/sub-nottwil03_T1w.nii.gz sub-nottwil04/anat/sub-nottwil04_T1w.nii.gz sub-nottwil05/anat/sub-nottwil05_T1w.nii.gz sub-nottwil06/anat/sub-nottwil06_T1w.nii.gz sub-nwu01/anat/sub-nwu01_T1w.nii.gz sub-nwu02/anat/sub-nwu02_T1w.nii.gz sub-nwu03/anat/sub-nwu03_T1w.nii.gz sub-nwu04/anat/sub-nwu04_T1w.nii.gz sub-nwu05/anat/sub-nwu05_T1w.nii.gz sub-nwu06/anat/sub-nwu06_T1w.nii.gz sub-oxfordFmrib01/anat/sub-oxfordFmrib01_T1w.nii.gz sub-oxfordFmrib02/anat/sub-oxfordFmrib02_T1w.nii.gz sub-oxfordFmrib03/anat/sub-oxfordFmrib03_T1w.nii.gz sub-oxfordFmrib04/anat/sub-oxfordFmrib04_T1w.nii.gz sub-oxfordFmrib05/anat/sub-oxfordFmrib05_T1w.nii.gz sub-oxfordFmrib06/anat/sub-oxfordFmrib06_T1w.nii.gz sub-oxfordFmrib07/anat/sub-oxfordFmrib07_T1w.nii.gz sub-oxfordFmrib08/anat/sub-oxfordFmrib08_T1w.nii.gz sub-oxfordFmrib09/anat/sub-oxfordFmrib09_T1w.nii.gz sub-oxfordFmrib10/anat/sub-oxfordFmrib10_T1w.nii.gz sub-oxfordFmrib11/anat/sub-oxfordFmrib11_T1w.nii.gz sub-oxfordOhba01/anat/sub-oxfordOhba01_T1w.nii.gz sub-oxfordOhba02/anat/sub-oxfordOhba02_T1w.nii.gz sub-oxfordOhba03/anat/sub-oxfordOhba03_T1w.nii.gz sub-oxfordOhba04/anat/sub-oxfordOhba04_T1w.nii.gz sub-oxfordOhba05/anat/sub-oxfordOhba05_T1w.nii.gz sub-pavia01/anat/sub-pavia01_T1w.nii.gz sub-perform01/anat/sub-perform01_T1w.nii.gz sub-perform02/anat/sub-perform02_T1w.nii.gz sub-perform03/anat/sub-perform03_T1w.nii.gz sub-perform04/anat/sub-perform04_T1w.nii.gz sub-perform05/anat/sub-perform05_T1w.nii.gz sub-perform06/anat/sub-perform06_T1w.nii.gz sub-queensland01/anat/sub-queensland01_T1w.nii.gz sub-queensland02/anat/sub-queensland02_T1w.nii.gz sub-queensland03/anat/sub-queensland03_T1w.nii.gz sub-queensland04/anat/sub-queensland04_T1w.nii.gz sub-queensland05/anat/sub-queensland05_T1w.nii.gz sub-queensland06/anat/sub-queensland06_T1w.nii.gz sub-sherbrooke01/anat/sub-sherbrooke01_T1w.nii.gz sub-sherbrooke02/anat/sub-sherbrooke02_T1w.nii.gz sub-sherbrooke03/anat/sub-sherbrooke03_T1w.nii.gz sub-sherbrooke04/anat/sub-sherbrooke04_T1w.nii.gz sub-sherbrooke05/anat/sub-sherbrooke05_T1w.nii.gz sub-sherbrooke06/anat/sub-sherbrooke06_T1w.nii.gz sub-sherbrooke07/anat/sub-sherbrooke07_T1w.nii.gz sub-stanford01/anat/sub-stanford01_T1w.nii.gz sub-stanford02/anat/sub-stanford02_T1w.nii.gz sub-stanford03/anat/sub-stanford03_T1w.nii.gz sub-stanford04/anat/sub-stanford04_T1w.nii.gz sub-stanford05/anat/sub-stanford05_T1w.nii.gz sub-stanford06/anat/sub-stanford06_T1w.nii.gz sub-strasbourg01/anat/sub-strasbourg01_T1w.nii.gz sub-strasbourg02/anat/sub-strasbourg02_T1w.nii.gz sub-strasbourg03/anat/sub-strasbourg03_T1w.nii.gz sub-strasbourg04/anat/sub-strasbourg04_T1w.nii.gz sub-strasbourg05/anat/sub-strasbourg05_T1w.nii.gz sub-strasbourg06/anat/sub-strasbourg06_T1w.nii.gz sub-tokyo750w01/anat/sub-tokyo750w01_T1w.nii.gz sub-tokyo750w02/anat/sub-tokyo750w02_T1w.nii.gz sub-tokyo750w03/anat/sub-tokyo750w03_T1w.nii.gz sub-tokyo750w04/anat/sub-tokyo750w04_T1w.nii.gz sub-tokyo750w05/anat/sub-tokyo750w05_T1w.nii.gz sub-tokyo750w06/anat/sub-tokyo750w06_T1w.nii.gz sub-tokyo750w07/anat/sub-tokyo750w07_T1w.nii.gz sub-tokyoIngenia01/anat/sub-tokyoIngenia01_T1w.nii.gz sub-tokyoIngenia02/anat/sub-tokyoIngenia02_T1w.nii.gz sub-tokyoIngenia03/anat/sub-tokyoIngenia03_T1w.nii.gz sub-tokyoIngenia04/anat/sub-tokyoIngenia04_T1w.nii.gz sub-tokyoIngenia05/anat/sub-tokyoIngenia05_T1w.nii.gz sub-tokyoIngenia06/anat/sub-tokyoIngenia06_T1w.nii.gz sub-tokyoIngenia07/anat/sub-tokyoIngenia07_T1w.nii.gz sub-tokyoSkyra01/anat/sub-tokyoSkyra01_T1w.nii.gz sub-tokyoSkyra02/anat/sub-tokyoSkyra02_T1w.nii.gz sub-tokyoSkyra03/anat/sub-tokyoSkyra03_T1w.nii.gz sub-tokyoSkyra04/anat/sub-tokyoSkyra04_T1w.nii.gz sub-tokyoSkyra05/anat/sub-tokyoSkyra05_T1w.nii.gz sub-tokyoSkyra06/anat/sub-tokyoSkyra06_T1w.nii.gz sub-tokyoSkyra07/anat/sub-tokyoSkyra07_T1w.nii.gz sub-ubc01/anat/sub-ubc01_T1w.nii.gz sub-ubc02/anat/sub-ubc02_T1w.nii.gz sub-ubc03/anat/sub-ubc03_T1w.nii.gz sub-ubc04/anat/sub-ubc04_T1w.nii.gz sub-ubc05/anat/sub-ubc05_T1w.nii.gz sub-ubc06/anat/sub-ubc06_T1w.nii.gz sub-unf01/anat/sub-unf01_T1w.nii.gz sub-unf02/anat/sub-unf02_T1w.nii.gz sub-unf03/anat/sub-unf03_T1w.nii.gz sub-unf04/anat/sub-unf04_T1w.nii.gz sub-unf05/anat/sub-unf05_T1w.nii.gz sub-unf06/anat/sub-unf06_T1w.nii.gz sub-unf07/anat/sub-unf07_T1w.nii.gz sub-vallHebron01/anat/sub-vallHebron01_T1w.nii.gz sub-vallHebron02/anat/sub-vallHebron02_T1w.nii.gz sub-vallHebron03/anat/sub-vallHebron03_T1w.nii.gz sub-vallHebron04/anat/sub-vallHebron04_T1w.nii.gz sub-vallHebron05/anat/sub-vallHebron05_T1w.nii.gz sub-vallHebron06/anat/sub-vallHebron06_T1w.nii.gz sub-vallHebron07/anat/sub-vallHebron07_T1w.nii.gz sub-vuiisAchieva01/anat/sub-vuiisAchieva01_T1w.nii.gz sub-vuiisAchieva02/anat/sub-vuiisAchieva02_T1w.nii.gz sub-vuiisAchieva03/anat/sub-vuiisAchieva03_T1w.nii.gz sub-vuiisAchieva04/anat/sub-vuiisAchieva04_T1w.nii.gz sub-vuiisAchieva05/anat/sub-vuiisAchieva05_T1w.nii.gz sub-vuiisAchieva06/anat/sub-vuiisAchieva06_T1w.nii.gz sub-vuiisIngenia01/anat/sub-vuiisIngenia01_T1w.nii.gz sub-vuiisIngenia02/anat/sub-vuiisIngenia02_T1w.nii.gz sub-vuiisIngenia03/anat/sub-vuiisIngenia03_T1w.nii.gz sub-vuiisIngenia04/anat/sub-vuiisIngenia04_T1w.nii.gz sub-vuiisIngenia05/anat/sub-vuiisIngenia05_T1w.nii.gz sub-vuiisIngenia06/anat/sub-vuiisIngenia06_T1w.nii.gz ```
renelabounek commented 2 years ago

@alexfoias Ok, thanks for the manual. I will clone the database tomorrow. On Thursday, I have meeting with Monica when we make visual inspection of updated scans and let you know.

renelabounek commented 2 years ago

@alexfoias: @mbondy023 and I have checked several scans. All appeared defaced fine with no loss in brain tissue volume. @mbondy023 will re-calculate freesurfer analysis on these scans and simultaneously check the defacing quality of each updated scan. Please, would it be possible to enable her to edit this particular comment: https://github.com/spine-generic/data-multi-subject/issues/115#issuecomment-1030220637? I was not able to give her the permission.

alexfoias commented 2 years ago

@kousu is it possible to give access to a edit right for this particular comment: #115 (comment) to a user ?

kousu commented 2 years ago

@kousu is it possible to give access to a edit right for this particular comment: #115 (comment) to a user ?

I don't think there's a way without granting membership, and we're pretty careful about permissions around this particular dataset.

Instead, I've got a basic workaround at https://github.com/spine-generic/data-multi-subject/issues/115#issuecomment-1065297685

alexfoias commented 2 years ago

@kousu thanks for the input.

alexfoias commented 2 years ago

@renelabounek I have pushed the new images on https://github.com/spine-generic/data-multi-subject/pull/119/commits/5e1d353a96c4824dc92e080f2cc18d25631e1594. Could you please review them ? thanks.

renelabounek commented 2 years ago

@renelabounek I have pushed the new images on https://github.com/spine-generic/data-multi-subject/commit/5e1d353a96c4824dc92e080f2cc18d25631e1594. Could you please review them ? thanks.

I have reviewed all cmrr scans and geneva04. All appears fine. Previous updates @mbondy023 approved. Regrading the comment https://github.com/spine-generic/data-multi-subject/issues/115#issuecomment-1065334918, many T1w scans with defacing issue remain. Should I start to email contributors? It does not appear like just a 2-week issue to fully finish it.

jcohenadad commented 2 years ago

@renelabounek I have pushed the new images on 5e1d353. Could you please review them ? thanks.

I have reviewed all cmrr scans and geneva04. All appears fine. Previous updates @mbondy023 approved. Regrading the comment #115 (comment), many T1w scans with defacing issue remain. Should I start to email contributors? It does not appear like just a 2-week issue to fully finish it.

I would not contact contributors, because many of them moved on to other projects/institutions and we might end up spending lots of time/efforts to hunt after them, and explain what is needed, what is defacing, explaining how to do it, and then review it ourselves, etc.

i think, timewise, it will be more effective if @alexfoias finishes up the defacing job (and @renelabounek if your team can help with that it would be awesome, maybe we could split the work)

alexfoias commented 2 years ago

for the remaining sites, we received the images already defaced. I have only pushed the images that we had on our server and which were received with the face.

jcohenadad commented 2 years ago

for the remaining sites, we received the images already defaced. I have only pushed the images that we had on our server and which were received with the face.

ok, so indeed, we need to contact the contributors, @renelabounek would you mind doing it? thanks!! 🙏

renelabounek commented 2 years ago

@jcohenadad I agree to close this PR but not the Issue #115. There is about 14 sites where are issues with the T1w defacing. I will email contributors of the particular sites with the request and cc you.

jcohenadad commented 2 years ago

@jcohenadad I agree to close this PR

I assume you meant 'merge and close' the PR (ie: we want to keep the changes made by @alexfoias). If that's the case, then yes, let's merge and close, and we will follow-up #115 in a subsequent PR.

renelabounek commented 2 years ago

@jcohenadad I agree to close this PR

I assume you meant 'merge and close' the PR (ie: we want to keep the changes made by @alexfoias). If that's the case, then yes, let's merge and close, and we will follow-up #115 in a subsequent PR.

Oh sorry, if you waited for ym response :-X Yes, that is what I meant.