bids-standard / bep001

Project management repository (primarily issues) for BIDS Extension Proposal 001
Creative Commons Attribution 4.0 International
8 stars 11 forks source link

Say No to symbolic links! not needed IMHO #82

Open yarikoptic opened 4 years ago

yarikoptic commented 4 years ago

Even with all my git-annex rotten soul I must say that BIDS must stay away from any filesystem features beyond having directories and files in them. And I think symbolic linking for some hypothetical purity is simply not needed here. One of my arguments is that BIDS itself, although intended for "raw" data, is containing "derived" data already in 99% of the cases:

so I do not think it is productive to try making it 100% explicit by placing "minimally processed" data into derivatives/ and then symlinking to establish the origin. BIDS does even somewhat "allows" for it:

src/02-common-principles.md-BIDS in its current form is designed to harmonize and describe raw (unprocessed
src/02-common-principles.md:or minimally processed due to file format conversion) data. During analysis such

so if outside-of-scanner processing is needed to derive a file, it is really close to "file format conversion". If such processing is more than un-ambiguous conversion/computation which could be parametrized, then it should be done and reflected in "BIDS common derivatives" fashion (BEP003 PR), e.g. by using _desc- field and all provenance tracking (BasedOn introduced in https://github.com/bids-standard/bep001/pull/59) should be "worked out" on "common derivatives" level.

Gilles86 commented 4 years ago

Hi @yarikoptic ,

Thanks for your interest in BEP-001.

We have the following problem: In quantitative MRI, we often want to calculate a "quantitative map", that is based on a number of MR images. For example, in the case of MP2RAGE, we can get a T1 map and a T1-weighted "UNI"-image, without B0 and B1 inhomogenieties. Sometimes we calculate these maps ourselves. Sometimes, they "roll right out" of the scanner.

Importantly, we often want to use these quantitative maps in further processing. For example, the T1UNI-image can be used to replace the T1w images in many fMRI pipelines (although fmriprep currently crashes on the salt-and-pepper noise in low-signal areas) and T1map works suprisingly well as a fake T2w-image in e.g. freesurfer.

In the current way BIDS works, it seems there is no way for BIDS apps to use data from the derivatives folder as input. That means that we can't use our nice qMRI maps as input to BIDS pipelines. How do you propose we deal with this?

Cheers, Gilles

effigies commented 4 years ago

it seems there is no way for BIDS apps to use data from the derivatives folder as input.

I'm curious how this was determined. Is there some text in the specification that specifically prevents it?

fMRIPrep uses FreeSurfer derivatives found in the output directory (which may or may not be <bids-root>/derivatives). FitLins takes -d <deriv> flags to add derivatives to index.

Or do you mean that the existing pipelines will not know to index derivatives? Given that a derivative dataset may include any source data it chooses, one option would be simply to make a directory that contains these a conformant derivatives dataset.

Gilles86 commented 4 years ago

Or do you mean that the existing pipelines will not know to index derivatives? Given that a derivative dataset may include any source data it chooses, one option would be simply to make a directory that contains these a conformant derivatives dataset.

Can you elaborate a bit on how this would work for the MP2RAGE/T1UNI example?