Allow other suffixes than `-manual`

[valosekj]

Sometimes, it might be useful to be able to specify other suffixes than -manual; for example -manual_rater1 or -manual_rater2

https://github.com/spinalcordtoolbox/manual-correction/blob/39785a65104ab53f5d1477fc06725c772a642358/manual_correction.py#L650-L652

Related issue: https://github.com/spinalcordtoolbox/manual-correction/issues/40

[NathanMolinier]

I wonder if the simple addition of -manual is BIDS compliant

See https://intranet.neuro.polymtl.ca/data/dataset-curation.html#derivatives-structure

[jcohenadad]

IMO I would not use the suffix "manual" anymore, because:

• this information is included in the JSON as per previous discussion, where the software OR the rater would be listed in the JSON sidecar
• in general, to train a DL model, we generate GT by running a pipeline with SOTA software and then manually correct a fraction of it (typically 10-20%). Which means, that the remaining 80-90% are NOT human-corrected, BUT should still be used for DL training.
  • A potential issue with this approach however, is that we need a mechanism to know if a label created by a software has already been reviewed by a human. The 'manual' suffix addresses this problem...
• The script that trains a DL model would not have to deal with the "-manual" or "manual2", etc. suffixes, but would only search for the type of the label, eg: label-SC_seg, or in the case of multiple raters (see below), label-SC_seg*

[jcohenadad]

I agree with https://github.com/spinalcordtoolbox/manual-correction/issues/48#issuecomment-1629296787 about mentioning that a label folders was either manually corrected and/or reviewed by a human (eg: called "labels_manual"). In case there is more than one rater, we could simply at a numerical suffix to the seg suffix. Examples:

```
...
...
├── sub-XXX
│   └── anat
│       └──sub-XXX_T1w.nii.gz
...
...
└── derivatives
    ├── dataset_description.json
    └── manual_labels
        ├── sub-XXX
        │   ├── anat
        │   │   ├──sub-XXX_T1w_label-SC_seg.nii.gz
        │   │   ├──sub-XXX_T1w_label-SC_seg1.nii.gz
        │   │   ├──sub-XXX_T1w_label-SC_seg2.nii.gz
        ...
        ...
```

[valosekj]

We had some additional in-person discussion with @NathanMolinier. We agree with removing the suffix -manual from label filenames.

However, we are a bit reluctant to rename derivatives/labels to derivatives/manual_labels for two reasons:

• the legacy reasons (all our datasets currently contain derivatives/labels)
• where would we put the automatically generated labels? Under a second directory (automatic_labels)?

We would stay just with derivatives/labels. This folder will contain both automatically generated as well as manually corrected files (as it is now). The information if the file was generated automatically or manually corrected will be stored in JSON sidecar (discussion here).

...
...
├── sub-XXX
│   └── anat
│       └──sub-XXX_T1w.nii.gz
...
...
└── derivatives
    ├── dataset_description.json
    └── labels
        ├── sub-XXX
        │   ├── anat
        │   │   ├──sub-XXX_T1w_label-SC_seg.nii.gz
        │   │   ├──sub-XXX_T1w_label-SC_seg.json            # will contain information on how it was generated (https://github.com/spinalcordtoolbox/spinalcordtoolbox/issues/3394), and if manually corrected, also the name 
        ...

[valosekj]

In case there is more than one rater, we could simply at a numerical suffix to the seg suffix. Examples:

Good idea!

[jcohenadad]

However, we are a bit reluctant to rename derivatives/labels to derivatives/manual_labels for two reasons:

i'm OK with that, but in that case we should be consistent and update our current SOP

where would we put the automatically generated labels? Under a second directory (automatic_labels)?

For us, there is often no clear distinction between automatic and manual labels, because labels are first created automatically, and then reviewed manually. The assumption should be that, if there is something under derivatives/ that has been git-annexed, then it has been reviewed by a human and it is correct (whether it was created automatically or not)

[valosekj]

However, we are a bit reluctant to rename derivatives/labels to derivatives/manual_labels for two reasons:

i'm OK with that, but in that case we should be consistent and update our current SOP

Okay! Updated in https://github.com/neuropoly/intranet.neuro.polymtl.ca/commit/729bc53ec57374a6e8305bac310399110cf4f0aa.

---

Also, regarding my original comment:

Sometimes, it might be useful to be able to specify other suffixes than -manual; for example -manual_rater1 or -manual_rater2

This can be actually achieved by already existing -suffix-files-seg flag. For example, we can use -suffix-files-seg '_seg1'.

--> closing the issue