Open yjmantilla opened 2 years ago
Regarding:
"This hints to me that it would be useful to have a way to get the files that match the pattern and acting just on them (and of course informing the user that there are files that didnt match the pattern if that is the case).
Now, whether this is implemented from different rules files that apply to the same sourcepath/dataset or rather a single rules file that handles multiple versions of the same rule is in discussion."
It is worth taking the BIDScoin example. BIDScoin indeed only converts the files that match a pattern in the rules file, but from what I saw, it does not inform the user of "unused" files. I actually think that informing the user of data files that were not included in the BIDS is important, as it can prevent mistakes in the conversion process.
Also, BIDScoin uses a single rules file that contains multiple patterns. That might or might not be the best approach for us.
I just want to point out that the comparison to the DICOM conversion in BIDScoin has its limitations. In DICOM, each sequence (anat, func, dwi) is matched using a different pattern, so we naturally have many patterns for each study. That might not be the case in EEG/MEG.
Regarding:
"Right now it is only seeing a collection of data files. I would prefer for it to be more general and not depend on the concept of participant, or maybe only slightly (like files having a similar sub-XXXX pattern)."
I'm not sure if BIDScoin has a notion of participant or not, but I do know that it take into account participants in incremental conversion. If a participant already has a subdirectory under the target BIDS folder, and the subdirectory already contain data for at least one sequence type (for example, anat), BIDScoin will not try to convert anything for that participant unless the -f flag is used.
I actually think that informing the user of data files that were not included in the BIDS is important, as it can prevent mistakes in the conversion process.
Agreed
I'm not sure if BIDScoin has a notion of participant or not, but I do know that it take into account participants in incremental conversion.
I think it does from what I remember of my last reading of the code. This can be seen here
mne-bids does have the concept of participant into account as well why is why I think we may not need it. I still need to better study the deal with incremental conversion to be sure.
Im copying this from a concern from @civier
What follows is my discussion with Oren regarding this topic, so that it does not get lost in a particular email:
I think acting on the same target bidspath is not the problem but rather acting on the same sourcepath which is scanned to get the files. I think this is what you clarified in the last message. This hints to me that it would be useful to have a way to get the files that match the pattern and acting just on them (and of course informing the user that there are files that didnt match the pattern if that is the case).
Now, whether this is implemented from different rules files that apply to the same sourcepath/dataset or rather a single rules file that handles multiple versions of the same rule is in discussion.
I would have to meditate this, and maybe the final conclusion will be achieved when we actually do the implementation. Nevertheless my intuition is that it would be identical to how it will treat different naming conventions in DIFFERENT participants as you said.
Right now it is only seeing a collection of data files. I would prefer for it to be more general and not depend on the concept of participant, or maybe only slightly (like files having a similar sub-XXXX pattern).
I think @DavidjWhite33 will have to help us on this. I dont think this use-case is common. From what I have seen, usually eeg files are provided without any metadata files apart from what the acquisition software provides.