search

ABCD-STUDY / nda-abcd-collection-3165

This is the documentation repository for ongoing releases to the NDA collection 3165. Please add GitHub Issues here for requests or feedback.
https://collection3165.readthedocs.io
BSD 3-Clause "New" or "Revised" License
16 stars 3 forks source link

Please provide more detail about sample exclusions. #47

Open mrpeverill opened 1 year ago

mrpeverill commented 1 year ago

I am having some trouble making sense of exclusions from the collection, which I need for a project evaluating QC pipelines. There are two parts to this issue:

  1. The documentation suggests that inclusion (i.e., "collection_3165" variable in participants.tsv is 1) is calculated from fields in the abcd_fastqc.txt data file. However, I haven’t been able to reproduce this list. I tried to use code from abcd-dicom2bids to process the fastqc file downloaded from release 2 of ABCD on the NDA. The output I get from those scripts appears to list >1000 participants as having valid T1, field map, and rsFMRI who are not listed as ‘in collection’ in the ‘participants.tsv’ file distributed with ABCC, and vice versa. Could you please provide steps to reproduce the 'collection_3165' field in participants.tsv? Part of the problem is that it is unclear which version of abcd-dicom2bids was used to filter the dataset, and the relevant code has changed over the history of the repository.
  2. The list of participants I was able to download rsFMRI data for and the list of participants in derivatives_qc.tsv is just about the same. However, I'm not sure how derivatives_qc was generated. It seems likely that participants listed as in 'collection_3165" were then subjected to further QC and that some were excluded -- can you provide information about how or why this was done? I cannot find any QC reports for the missing participants.

Let me know if you need more information. Thank you for a great collection!

ericfeczko commented 1 year ago

Hi @mrpeverill!

Unfortunately, the ABCD FastTrack QC is continually updated. New datasets that were rejected may appear as acceptable after revisions, and/or sometimes new data are added. It is important to make sure that the fidelity of the ABCD FastTrack QC is matched to the release.

We should have that maintained for subsequent releases, @hough129 may be able to provide more information.

mrpeverill commented 1 year ago

@ericfeczko Thank you that would be really helpful! It makes sense that fastqc would become out of date over time. Can you tell me if any participants were excluded from your own QC processes? I'm specifically curious if you excluded anyone because of their motion parameters, because the amount of motion in currently released participants is lower than I expected.