search

sibis-platform / ncanda-data-integration

This is the Data Integration, MRI, and Bioinformatics Component of the National Consortium on Alcohol and NeuroDevelopment in Adolescence (NCANDA), funded by the NIAAA.
https://www.nitrc.org/projects/ncanda-datacore
BSD 3-Clause "New" or "Revised" License
4 stars 10 forks source link

import_mr_sessions --event doesn't consider other events #358

Open shippy opened 4 years ago

shippy commented 4 years ago

As described in #357:

import_mr_sessions assumes that every visit with a visit date on any arm is, or can be, associated with a scan. The other arms often do have a date that follows the main visit closely, and a scan that comes along with it.

To avoid assigning those non-Arm-1 scans to the main arm, import_mr_sessions looks for the next visit date and shortens the lookup period if it's within 120 days of main arm visit.

But this doesn't happen if the --event argument is specified. args.event is used to constrain the data export prior to the determination of scan lookup period.

Resolution: Always export all events and keep them in visit_log_redcap, but inside the iterrows() loop, check the redcap_event_name of the current row and continue if it doesn't match args.event.

Replication: D-00155-M-0 has a Year 5 visit that's followed by a Recovery Baseline. Everything works if redcap/import_mr_sessions -vf --study-id D-00155-M-0 is invoked, but:


#ncanda@ncanda[pipeline-front]:/sibis-software/ncanda-data-integration/scripts (master)$
redcap/import_mr_sessions -vf --study-id D-00155-M-0 -n --event 5y_visit_arm_1

Namespace(event='5y_visit_arm_1', force_update=True, force_update_stroop=False, 
  max_days_after_visit=120, missing_only=False, no_stroop=False, no_upload=True, 
  pipeline_root_dir=None, post_to_github=False, run_pipeline_script=None, 
  site=None, study_id='D-00155-M-0', time_log_dir=None, verbose=True)
Checking 1 REDCap records.
1 Processing ('D-00155-M-0', '5y_visit_arm_1')
Checking D-00155-M-0 for 5y_visit_arm_1 with visit date 2018-04-23 to 2018-08-21
XNAT Session Data: [('NCANDA_E07827', '<ohsu_incoming>', '2018-04-30'), 
                    ('NCANDA_E07926', '<ohsu_incoming>', '2018-05-30')]
{"experiment_site_id": "D-00155-M-0-2018-04-23", 
 "error": "ERROR: more than one scan of type mprage on the same visit ",
 "eid_and_scan_number": "[('NCANDA_E07827', '3', 7), ('NCANDA_E07926', '3', 37)]",
 "type": "mprage"}
{"experiment_site_id": "D-00155-M-0-2018-04-23",
 "error": "ERROR: more than one scan of type t2fse on the same visit ",
 "eid_and_scan_number": "[('NCANDA_E07827', '2', 7), ('NCANDA_E07926', '2', 37)]",
 "type": "t2fse"}
{"experiment_site_id": "D-00155-M-0-2018-04-23",
 "error": "ERROR: more than one scan of type dti6b500pepolar on the same visit ",
 "eid_and_scan_number": "[('NCANDA_E07827', '4', 7), ('NCANDA_E07926', '4', 37)]",
 "type": "dti6b500pepolar"}
{"experiment_site_id": "D-00155-M-0-2018-04-23",
 "error": "ERROR: more than one scan of type dti30b400 on the same visit ",
 "eid_and_scan_number": "[('NCANDA_E07827', '12', 7), ('NCANDA_E07926', '12', 37)]",
 "type": "dti30b400"}
{"experiment_site_id": "D-00155-M-0-2018-04-23",
 "error": "ERROR: more than one scan of type dti60b1000 on the same visit ",
 "eid_and_scan_number": "[('NCANDA_E07827', '6', 7), ('NCANDA_E07926', '6', 37)]",
 "type": "dti60b1000"}
{"experiment_site_id": "D-00155-M-0-2018-04-23",
 "error": "ERROR: more than one scan of type rsfmri on the same visit ",
 "eid_and_scan_number": "[('NCANDA_E07827', '20', 7), ('NCANDA_E07926', '20', 37)]",
 "type": "rsfmri"}
check_for_stroop: ['NCANDA_E07827', 'NCANDA_E07926']
check_for_stroop: no stroop
Suppressed uploading of 1 records to REDCap.