Option to reset studyspec.json

[manuelakuhn]

While developing a rule file I came across the problem that there seems to be no possibility to reset the studyspec.json file. Writing own rule files is quite some back and forth, especially if it is the first time to do it, but because the studyspec file always builds on top of the already existing one the behaviour can be unexpected (and sometimes not revertible).

An option to reset the studyspec to the state it was before any rule was applied would help here.

Unfortunately I do not know anymore what I did exactly when I first encounter this problem but I was able to create a (somewhat artificial) minimal example:

$ datalad create -c hirni sourcedata
$ cd sourcedata
$ datalad hirni-import-dcm --anon-subject 001 <data path>/sourcedata.tar.gz acq1
$ cd ..

$ datalad create -c bids bids
$ cd bids
$ datalad install --dataset . --source ../sourcedata sourcedata --recursive
$ cd sourcedata
$ mkdir code/custom_rules
$ wget -P code/custom_rules https://raw.githubusercontent.com/psychoinformatics-de/datalad-hirni/master/datalad_hirni/resources/rules/custom_rules_template.py
$ cat << EOT >> .datalad/config
[datalad "hirni.dicom2spec"]
        rules = code/custom_rules/custom_rules_template.py
EOT

1. create rule + apply rule

   def _rules(self, series_dict, subject=None, anon_subject=None,
              session=None):
   
       desc = series_dict['SeriesDescription']
   
       modality = None
       if desc.startswith("t1"):
           modality = "T1w"
   
       return {'description': series_dict['SeriesDescription']
               if "SeriesDescription" in series_dict else '',
   
               'comment': 'I actually have no clue',
               'subject': series_dict['PatientID'] if not subject else subject,
               'anon-subject': anon_subject if anon_subject else None,
               'bids-session': session if session else None,
               'bids-modality': modality
               }

   $ datalad hirni-dicom2spec -s acq1/studyspec.json acq1/dicoms
   $ cd ..
   $ datalad hirni-spec2bids --anonymize sourcedata/studyspec.json sourcedata/*/studyspec.json

   -> result

   $ tree sub-001 | sed "s/-> .*//"
   sub-001
   ├── anat
   │   ├── sub-001_run-1_T1w.json
   │   └── sub-001_run-1_T1w.nii.gz
   ├── sub-001_scans.json
   └── sub-001_scans.tsv
   
   1 directory, 4 files

2. change rule by adding run

                    'bids-modality': modality,
   +                'bids-run': None
                    }

   clean up and recreate bids:

   $ rm -rf sub-001
   $ cd sourcedata
   $ datalad hirni-dicom2spec -s acq1/studyspec.json acq1/dicoms
   $ cd ..
   $ datalad hirni-spec2bids --anonymize sourcedata/studyspec.json sourcedata/*/studyspec.json

   -> file names change

   $ tree sub-001 | sed "s/-> .*//"
   sub-001
   ├── anat
   │   ├── sub-001_T1w.json
   │   └── sub-001_T1w.nii.gz
   ├── sub-001_scans.json
   └── sub-001_scans.tsv
   
   1 directory, 4 files

3. revert back changes in rule file + try again:

                    'bids-modality': modality,
   -                'bids-run': None
                    }

   -> output not reverted, would expect same output as in 1. but got output same as 2.:

   $ tree sub-001 | sed "s/-> .*//"
   sub-001
   ├── anat
   │   ├── sub-001_T1w.json
   │   └── sub-001_T1w.nii.gz
   ├── sub-001_scans.json
   └── sub-001_scans.tsv
   
   1 directory, 4 files

[bpoldrack]

Hm.

While developing a rule file I came across the problem that there seems to be no possibility to reset the studyspec.json file.

What about git checkout HEAD~1 -- studyspec.json (or whatever state you want to reset to)? May be I miss something, but not clear to me what else would be required/desired.

[manuelakuhn]

Although this is doable I see multiple problems with it:

1) You always have to keep track when you did what, especially if you do something else in between or find out that your rule does not quite fit way later in data taking and converting. To give a more drastic example: When a problem is introduced in some older commit, and afterwards some correct rule additions where made, a simple checkout would be not enough. 2) In addition this is quite inconvenient. The number of steps needed is already quite large compared to other bids conversion tools. 3) Especially concerning inexperienced users having to revert to an older commit seems overly complicated to me.

I was more thinking of a flag to give to dicom2spec with which the studyspec can be reset. Something like

$ datalad hirni-dicom2spec --reset -s acq1/studyspec.json acq1/dicoms

would create the studyspec anew.

[bpoldrack]

Okay. So, not reset to some previous state, but just recreate from scratch. Yes, that should be easy and useful.