search

spine-generic / data-multi-subject_DO-NOT-USE

Multi-subject data for the Spine Generic project
0 stars 1 forks source link

implement CI check_data_consistency #27

Open alexfoias opened 4 years ago

alexfoias commented 4 years ago

Fixes https://github.com/spine-generic/data-multi-subject/issues/24

alexfoias commented 4 years ago

Output from current master:

Warning missing following subjects from participants.tsv: 
['sub-fslAchieva01',
 'sub-fslAchieva02',
 'sub-fslAchieva03',
 'sub-fslAchieva04',
 'sub-fslAchieva05',
 'sub-fslAchieva06',
 'sub-fslPrisma01',
 'sub-fslPrisma02',
 'sub-fslPrisma03',
 'sub-fslPrisma04',
 'sub-fslPrisma05',
 'sub-fslPrisma06']

Warning missing data for subjects listed in participants.tsv: 
['sub-sapienza01',
 'sub-sapienza02',
 'sub-sapienza03',
 'sub-sapienza04',
 'sub-sapienza05',
 'sub-sapienza06']
Missing jsonSidecar: ./derivatives/labels/sub-mgh01/anat/sub-mgh01_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-cmrra01/anat/sub-cmrra01_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-vuiisIngenia05/anat/sub-vuiisIngenia05_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-milan01/anat/sub-milan01_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-milan01/anat/sub-milan01_T1w_RPI_r_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-vuiisIngenia03/anat/sub-vuiisIngenia03_T2star_rms_gmseg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-sherbrooke03/anat/sub-sherbrooke03_T2star_rms_gmseg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-tokyo750w02/anat/sub-tokyo750w02_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-tokyo750w02/anat/sub-tokyo750w02_T1w_RPI_r_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-tokyo750w05/anat/sub-tokyo750w05_T1w_RPI_r_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-tokyo750w05/anat/sub-tokyo750w05_T2star_rms_gmseg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-tehranS03/anat/sub-tehranS03_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-stanford05/anat/sub-stanford05_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-stanford02/anat/sub-stanford02_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-mpicbs02/anat/sub-mpicbs02_T1w_RPI_r_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-tehranS02/anat/sub-tehranS02_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-strasbourg01/anat/sub-strasbourg01_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-tokyo750w03/anat/sub-tokyo750w03_T1w_RPI_r_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-mniS05/anat/sub-mniS05_T2w_RPI_r_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-mniS05/anat/sub-mniS05_T2star_rms_gmseg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-stanford03/anat/sub-stanford03_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-stanford04/anat/sub-stanford04_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-stanford04/anat/sub-stanford04_T1w_RPI_r_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-barcelona05/anat/sub-barcelona05_T2w_csfseg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-brno08/anat/sub-brno08_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-brno08/anat/sub-brno08_T1w_RPI_r_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-vallHebron07/anat/sub-vallHebron07_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-brno06/anat/sub-brno06_T1w_RPI_r_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-beijingVerio02/anat/sub-beijingVerio02_T2w_RPI_r_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-dresden02/anat/sub-dresden02_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-brno01/anat/sub-brno01_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-mniPilot1/anat/sub-mniPilot1_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-perform06/anat/sub-perform06_T1w_RPI_r_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-perform06/anat/sub-perform06_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-balgrist02/anat/sub-balgrist02_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-beijingVerio03/anat/sub-beijingVerio03_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-brno07/anat/sub-brno07_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-brno07/anat/sub-brno07_T1w_RPI_r_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-oxfordFmrib09/anat/sub-oxfordFmrib09_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-nottwil03/anat/sub-nottwil03_T2star_rms_gmseg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-nottwil03/anat/sub-nottwil03_T2w_csfseg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-nottwil04/anat/sub-nottwil04_T1w_RPI_r_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-nottwil04/anat/sub-nottwil04_T2w_csfseg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-oxfordFmrib07/anat/sub-oxfordFmrib07_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-queensland03/anat/sub-queensland03_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-fslAchieva05/anat/sub-fslAchieva05_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-geneva04/anat/sub-geneva04_T2w_csfseg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-oxfordOhba05/anat/sub-oxfordOhba05_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-oxfordOhba05/anat/sub-oxfordOhba05_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-tokyoIngenia01/anat/sub-tokyoIngenia01_T2star_rms_gmseg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-oxfordFmrib01/anat/sub-oxfordFmrib01_T1w_RPI_r_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-tokyoIngenia06/anat/sub-tokyoIngenia06_T2w_csfseg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-oxfordFmrib06/anat/sub-oxfordFmrib06_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-nottwil05/anat/sub-nottwil05_T1w_RPI_r_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-oxfordFmrib08/anat/sub-oxfordFmrib08_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-nwu06/anat/sub-nwu06_T2w_RPI_r_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-nwu01/anat/sub-nwu01_T1w_RPI_r_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-oxfordOhba03/anat/sub-oxfordOhba03_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-fslAchieva03/anat/sub-fslAchieva03_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-fslAchieva04/anat/sub-fslAchieva04_T2star_rms_gmseg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-fslAchieva04/anat/sub-fslAchieva04_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-cmrra05/anat/sub-cmrra05_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-pavia05/anat/sub-pavia05_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-sherbrooke01/anat/sub-sherbrooke01_T2star_rms_gmseg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-pavia02/anat/sub-pavia02_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-mgh05/anat/sub-mgh05_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-mgh02/anat/sub-mgh02_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-pavia03/anat/sub-pavia03_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-pavia03/anat/sub-pavia03_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-pavia04/anat/sub-pavia04_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-pavia04/anat/sub-pavia04_T2w_csfseg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-oxfordFmrib10/anat/sub-oxfordFmrib10_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-beijingPrisma01/anat/sub-beijingPrisma01_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-stanford06/anat/sub-stanford06_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-stanford06/anat/sub-stanford06_T2star_rms_gmseg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-stanford06/anat/sub-stanford06_T1w_RPI_r_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-stanford06/anat/sub-stanford06_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-mpicbs01/anat/sub-mpicbs01_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-tokyo750w06/anat/sub-tokyo750w06_T1w_RPI_r_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-tokyo750w06/anat/sub-tokyo750w06_T2star_rms_gmseg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-tokyo750w01/anat/sub-tokyo750w01_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-mpicbs07/anat/sub-mpicbs07_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-cardiff05/anat/sub-cardiff05_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-mniS01/anat/sub-mniS01_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-mniS08/anat/sub-mniS08_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-tehranS01/anat/sub-tehranS01_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-vuiisAchieva01/anat/sub-vuiisAchieva01_T2star_rms_gmseg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-vuiisAchieva01/anat/sub-vuiisAchieva01_T1w_RPI_r_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-tehranS06/anat/sub-tehranS06_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-beijingGE04/anat/sub-beijingGE04_T1w_RPI_r_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-vallHebron03/anat/sub-vallHebron03_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-perform04/anat/sub-perform04_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-tokyoSkyra03/anat/sub-tokyoSkyra03_T2star_rms_gmseg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-tokyoSkyra03/anat/sub-tokyoSkyra03_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-tokyoSkyra03/anat/sub-tokyoSkyra03_T1w_RPI_r_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-tokyoSkyra03/anat/sub-tokyoSkyra03_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-brno02/anat/sub-brno02_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-brno02/anat/sub-brno02_T1w_RPI_r_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-brno05/anat/sub-brno05_T1w_RPI_r_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-brno05/anat/sub-brno05_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-tokyoSkyra04/anat/sub-tokyoSkyra04_T2star_rms_gmseg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-amu02/anat/sub-amu02_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-amu05/anat/sub-amu05_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-balgrist01/anat/sub-balgrist01_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-perform02/anat/sub-perform02_T2w_csfseg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-perform02/anat/sub-perform02_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-brno04/anat/sub-brno04_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-brno03/anat/sub-brno03_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-brno03/anat/sub-brno03_T1w_RPI_r_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-amu04/anat/sub-amu04_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-cmrrb03/anat/sub-cmrrb03_T2w_csfseg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-cmrrb03/anat/sub-cmrrb03_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-fslAchieva01/anat/sub-fslAchieva01_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-fslAchieva01/anat/sub-fslAchieva01_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-fslAchieva01/anat/sub-fslAchieva01_T2star_rms_gmseg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-fslAchieva06/anat/sub-fslAchieva06_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-juntendo750w02/anat/sub-juntendo750w02_T1w_RPI_r_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-tokyoIngenia03/anat/sub-tokyoIngenia03_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-tokyoIngenia03/anat/sub-tokyoIngenia03_T2star_rms_gmseg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-tokyoIngenia03/anat/sub-tokyoIngenia03_T1w_RPI_r_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-oxfordFmrib03/anat/sub-oxfordFmrib03_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-tokyoIngenia04/anat/sub-tokyoIngenia04_T2star_rms_gmseg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-brnoPrisma04/anat/sub-brnoPrisma04_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-nwu02/anat/sub-nwu02_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-geneva01/anat/sub-geneva01_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-tokyoIngenia05/anat/sub-tokyoIngenia05_T2w_csfseg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-oxfordFmrib05/anat/sub-oxfordFmrib05_T1w_RPI_r_labels-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-tokyoIngenia02/anat/sub-tokyoIngenia02_T2star_rms_gmseg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-queensland01/anat/sub-queensland01_acq-T1w_MTS_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-nottwil01/anat/sub-nottwil01_T2star_rms_gmseg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-nottwil01/anat/sub-nottwil01_T2w_csfseg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-nottwil01/anat/sub-nottwil01_T1w_RPI_r_seg-manual.json
Missing jsonSidecar: ./derivatives/labels/sub-juntendo750w03/anat/sub-juntendo750w03_T1w_RPI_r_seg-manual.json
alexfoias commented 4 years ago

@jcohenadad This PR is ready for review.

alexfoias commented 4 years ago

Yes. I won't, I'll wait for your approval.

kousu commented 4 years ago

Sorry! You're going to have to remake this over in the remade https://github.com/spine-generic/data-multi-subject. And you'll have to add a git annex get line to the CI script, now (or will you? does the bids validator actually check that the niftis are valid images?).

I also would caution about running that on every CI run because that means downloading 10G to travis every time. That is going to make CI slow. Travis supports caching specific dirs: https://docs.travis-ci.com/user/caching/ but they have a warning that caching doesn't do much unless you're caching processing steps -- because each VM needs to download the data fresh each time; still it might be worth experimenting at least for an hour or two with. What if we only ran the validator in the nightly cronjob?

kousu commented 4 years ago

(or will you? does the bids validator actually check that the niftis are valid images?).

Answered my own question: https://travis-ci.com/github/spine-generic/data-multi-subject-2/builds/179667411. Hard yes:

1: [ERR] This file is too small to contain the minimal NIfTI header. (code: 36 - NIFTI_TOO_SMALL)

  ./sub-amu01/anat/sub-amu01_T1w.nii.gz
  ./sub-amu03/anat/sub-amu03_T2star.nii.gz
  ... and 1030 more files having this issue (Use --verbose to see them all).