search

spine-generic / data-multi-subject

Multi-subject data for the Spine Generic project
Creative Commons Attribution 4.0 International
22 stars 15 forks source link

Move derivatives from `spine-generic-processed` back to `data-multi-subject` #121

Closed sandrinebedard closed 2 years ago

sandrinebedard commented 2 years ago

Context

A processed version of this dataset was created --> spine-generic-processed because of incomptibility of the derivatives and the raw data (mainly for training) Related issue. The same strategy was used for UK-Biobank --> issue.

Problems with the -processedapproach

Note: This wasn't an issue for uk-biobank since there are no derivatives in the uk-biobank dataset, only in uk-biobank-processed

Solution: Move back all the derivatives to data-multi-subject

The proposed solution is to move back the derivatives in data-multi-subject. The incompatibilities will be addressed in a processing pipeline for each individual projects. The suffixes will be added back so the processing that was done is explicit.

Some concerns about moving it back:

Related issues:

jcohenadad commented 2 years ago

The suffixes _RPI_R ... are not BIDS compatible --> Maybe we don't keep them?

This is a good point indeed. For reference, here is the BIDS definition for derivatives. Another thing I am thinking of, is that in case the preprocessing script changes (eg: we don't reorient anymore), the derivative name will still have '_RPI' in it, which might cause confusion (because nowhere in the preprocessing script there will be a mention of this reorientation).

For the long term, some derivatives could be from various pipelines --> can get confusing when someone wants to use some derivatives and find the procesing script.

Good point, although I wouldn't worry too much at this point. I see more pros than cons, mostly driven by pragmatic concerns: we are currently struggling by not having the derivatives centralized, where we don't know if maybe within the next 5 years other derivatives will possibly conflict with the existing derivatives. I would say, if that happens, we will find a solution (eg: create another folder under derivatives, labels_pipelineX/)

sandrinebedard commented 2 years ago

This is a good point indeed. For reference, here is the BIDS definition for derivatives. Another thing I am thinking of, is that in case the preprocessing script changes (eg: we don't reorient anymore), the derivative name will still have '_RPI' in it, which might cause confusion (because nowhere in the preprocessing script there will be a mention of this reorientation).

Ok, so I will continue with the merging, but without any suffix, is that right (except -manual etc.)?