Data Monitoring | THRIVE

[jessb0t]

upgrade of data monitoring process to handle:

• multiple runs, multiple sessions
• multiple potential parental respondents associated with a single child
• English and Spanish parental responses
• multiple REDCap projects
• bidsify checked/

[jessb0t]

Initial Notes

• update hallMonitor to identify s1_r1, s2_r1, and s3_r1 embedded within each variable name and using this information to map its search for files in sourcedata/raw/sX ?
• update hallMonitor to identify a list of unique REDCap projects (e.g. thrive_iqs_child_s1_r1 and thrive_iqs_parent_s1_r1) so that it can identify the most recent version for each and then copy to sourcedata/checked accordingly?
• update update-tracker.py to look for any one of a trio of numbers (e.g., 300XXXX/308XXXX/309XXXX) when determining to which row an update should be applied? (Logic: the child participant is 300, so we want one row per child. But the questionnaires that their parents answer, which will be stored in separate REDCap .csv files, will be connected to the parent ID, which will be either 308 or 309.)
• update update-tracker.py to collapse across English and Spanish surveys (surveyname and surveynameES) so that the central tracker could be queried to see if data for a given survey exists, regardless of which language it was answered in. For this one, it could be useful to create a specific convention that informs update-tracker.py that it should collapse across the languages; for instance, if the variable it is given is something like “surveynameBOTH”.

[jessb0t]

Notes from 5/10 meeting

• each row is a child
• columns for the parent data (“1” means that either parent answered)
• 6 columns (one for each session) that indicate either (a) binary variable indicating whether primary parent answered or (b) id of the parent who answered questions
• option: 6 columns (one for each session) indicating in which language the parent answered questions

[jessb0t]

Notes from 5/15 meeting

• bbs_tracker: central tracker note about whether data exists
• iqs_tracker: central tracker note about whether data exists (plus specific questionnaires)
• need way to feed redcap project names during setup (so hallMonitor knows what to look for in each sx_rx)
• third bullet :point_up: : gb prefers “a”
• fourth bullet :point_up: : gb says go with 1=EN, 0=ES

After meeting, ja/dh discussed that there should be no duplicate column names across the REDCap files. To be safe, building a check that throws an error if it sees a duplicate.

[lillylaplace]

Notes from 6/1 meeting

• when updating scripts to work for multiple runs/sessions, do we want to maintain backwards compatibility with single-shot data collection projects that only have a single session & run? GB response: if it adds negligible additional time & complexity, go for it; if not, only concerned w it working for THRIVE, CASPER, & oops faces
• if it is built to be backwards compatible, that should be tested

[lillylaplace]

Note from JA on Slack on 6/5: We need to manipulate variable names when we set up hallMonitor for the first time. We will need to map consent_es_complete, which is what is in REDCap, to consentescomplete. In this way, David’s script can use the same logic to collapse across English and Spanish versions of REDCap consent data (based on whether the last two letters before the are “es”) in the same way he is doing for the survey data.

[davhunt]

For the "arrow-alert-vN" psychopy tasks, including the version number vN in the central tracker datadict when matching up with psychopy task file names and updating central tracker.

[davhunt]

For surveys that are given to both parent and child, specifying these surveys during the setup.sh call so that hallMonitor knows to map the child's survey to "[survey-name]_s1_r1_e1" and the parent's survey to something like "[survey-name]_parental_self_report_s1_r1_e1" when updating central tracker.

[davhunt]

When a survey has a version number (like masies_b_s1_r1_e1) we can assume it will always be a single letter? Or will it ever be a letter + number or something?

[jessb0t]

When a survey has a version number (like masies_b_s1_r1_e1) we can assume it will always be a single letter? Or will it ever be a letter + number or something?

@davhunt It will also be a set of letters (single letter until we hit "z", then "aa" and so on, never numbers or anything else).

[davhunt]

For backward- (and forward-) compatibility monitoring should be able to run on projects that

1. may or may not have both child and parent data, and
2. may or may not have both English and Spanish survey responses

These should either be specified by a flag during setup, or the monitoring scripts should detect whether these are true and behave accordingly (if for example it only sees IDs starting with 301, 308 and 309)

[jessb0t]

:point_up: comment immediately above discussed at meeting of 6/9:

1. researcher will specify with a flag at data monitoring setup
2. no need for the researcher to communicate; script will automatically look for any survey/surveyES and collapse accordingly

[lillylaplace]

Notes from DH/LL script testing on 6/19:

1. parent and consent are now being logged correctly in central tracker
2. parent identity (primary or secondary) and language (English or Spanish) are now being logged correctly in central tracker
3. 5 self-reports both parents and kids answer (bfne, fpe, masi, pints, via) are being handled by a flag in the setup script which creates new columns for parent self-reports at end of tracker which have the string "parent" appended into their names
   • problem: tracker is creating "parent"-appended columns for bfnep and fpep, which should only happen for bfne and fpe. DH rewriting script to look for "bfne" and "fpe" rather than just "bfne" and "fpe" so that parent reports don't get picked up
   • problem: tracker is maintaining columns for parent self-reports named "bfne_b_s1_r1_e1.1" and so on. DH rewriting script so that those columns are not created but instead are replaced with the "parent"-appended columns
4. scaared_b_s1_r1_e1 is empty in tracker even though there is data for that survey within raw

[lillylaplace]

Notes from DevOps meeting on 6/20:

1. DH fixed script so that parent self-report columns with "e1.1" names are no longer being created in central tracker
2. DH fixed script so that bfnep and fpep are no longer being treated as parent self-reports and being appended with "parent" and added in as columns at end of tracker
3. scaared_b_s1_r1_e1 column was empty in tracker because scaared was missing the "_b" in the central tracker data dictionary (occurred because the "_b" was at one point missing in the instrument's name in REDCap, but this was fixed)
   • NOTE FOR FUTURE TROUBLESHOOTING: when column shows up empty in tracker but there is data present for that survey in raw, first place to check is central tracker data dictionary
4. new method for handling 5 self-reports both parents and kids answer -- rather than using a flag in setup script, the names of the parent self-reports for those 5 instruments will be appended with "parent" within the central tracker data dictionary
   • ideally, this will be handled in the scripts in a way that is modular and can apply to future studies that won't specify "parent" but some other string of text. DH to set up this way if possible, or setup for THRIVE now and make modular later if too time-intensive

Next steps: test scripts rigorously (try to break them), then test with real data. After that, work with Emily Martin on setting up data monitoring for oops-faces

[lillylaplace]

Notes from Slack conversation on 6/14:

1. audio files will be named "audacity" --> scripts and data dictionary should be updated accordingly
2. video files will be named "zoom" --> scripts and data dictionary should be updated accordingly
3. audacity and zoom files will be named according to the following convention: [thriveID]_all-["single" or "double"]_zoom-_s1_r1_e1 and [thriveID]_all-["single" or "double"]_audacity-_s1_r1_e1; the "single" flag will be for files with one in-person child (and one remote child), the "double" flag will be for two in-person participants --> scripts and data dictionary should be updated accordingly
4. for participants who do not have data for a given file type due to experimental error (as opposed to missing data due to failure to upload), text files will be inserted in those file folders that say "no-digi" or "no-eeg" etc. --> DH confirms this will not cause problems for data monitoring scripts, which don't mind extra files as long as they are correctly nested within the established folder structure

[lillylaplace]

Up-to-date checklist of THRIVE data monitoring projects:

• scripts handle multiple runs and multiple sessions :white_check_mark:
• scripts handle multiple potential parental respondents associated with a single child :white_check_mark:
• scripts handle English and Spanish parental responses :white_check_mark:
• scripts handle multiple REDCap projects :white_check_mark:
• hallMonitor identifies the most recent version for each REDCap file then copies to sourcedata/checked :white_check_mark:
• tracker with one row per child, associated parent data in that same row (works regardless of whether parents answered in english or spanish) :white_check_mark:
• tracker with 6 rows (one for each session of iqs and bbs) that indicate whether primary or secondary parent answered (1 = primary, 0 = secondary) :white_check_mark:
• tracker with 6 rows (one for each session of iqs and bbs) that indicate which language the parent answered in (1 = English, 0 = Spanish) :white_check_mark:
• scripts handle files being called audio/audacity interchangeably :white_check_mark:
• scripts handle files being called video/zoom interchangeably :white_check_mark:
• audacity and zoom file naming convention ([thriveID]_all-["single" or "double"]_audacity_s1_r1_e1) does not cause problems for scripts :white_check_mark:
• text files that indicate data is missing due to experimental error ("no-digi", "no-eeg", etc.) do not cause problems for scripts :white_check_mark:
• sourcedata/checked in bids(ish) structure :white_check_mark:
• REDCap data for 5 self-reports both parents and kids answer are handled properly (parent self-reports are appended with "parent" label within central tracker data dictionary and central tracker) :white_check_mark:
• "consent_es_complete" changed to "consentes_complete" so hallMonitor can read it :white_check_mark:
• variable in tracker that notes whether data exists for iqs_clinician :white_check_mark:
• variable in tracker that notes whether data exists for bbs_RA project :white_check_mark:
• check built in that creates error if there are duplicate column names across the REDCap files :white_check_mark:
• scripts identify s#_r1 in each variable name and use this to search for files in sourcedata/raw/s# :white_check_mark:

[lillylaplace]

Notes from meeting 7/10:

• create error log file that can be viewed after running hallMonitor --> nice-to-have; not necessary right now (.out file is enough)
• in cases where a folder has extra files, scripts should report error unless there is a "corrected.txt" file then it shouldn't report an error but also shouldn't move files to checked
• set up preprocessing script
• need to test our fix for the score names of the 5 parent self-reports that kids also answer

[lillylaplace]

Notes from meeting 7/17:

• data dictionary variables for zoom, audacity, eeg, digi should use file naming conventions --> scripts need to be updated to accommodate this change
• scripts should be checking that files are named properly and there is the correct amount of files in a given folder (right now just checks that folder has data and that it's encrypted for audacity and zoom data)

[lillylaplace]

To-do list from meeting 7/19:

• scripts should check that files within a given data type folder are named properly based on the variable name associated with that data type within the central tracker data dictionary
• scripts should know how many files to expect within a given data type folder (as well as what naming convention to expect) and error-out when there are more or less than that number of files within a folder of that data type (or if the file names are incorrect) --> if there is a text file called "corrected.txt" within that folder, scripts will no longer give an error message; this should apply to all data types (psychopy, audacity, zoom, eeg, digi)
• scripts should read the allowedSuffix in the central tracker data dictionary and check that file names include the correct suffix and are nested within the associated folder (s1_r1, s2_r1, or s3_r1)

[lillylaplace]

Updates from meeting 7/27:

• [x] Script is now looking for file names that match the correct naming convention based on the variable name ("subject_variablename_suffix.ext") and erroring if it doesn't find all the expected file names
• Currently not erroring if file with incorrect name is found, but this should be built in before Wednesday's DevOps
• If file with incorrect name is found but a "corrected.txt" file is present, scripts will not error but also not copy (this is partially built in as of now -- scripts are looking for file w name that starts with "corrected" and not erroring if it's present, but script is not currently erroring when incorrect file name is present)
• [x] Script is also looking for files with the correct suffix based on the allowedSuffix column
• Currently not erroring if file with incorrect suffix is found, but this should be built in before Wednesday's DevOps
• [x] David added in a column to the data dictionary that tells the script what types of file extensions to expect for a given variable -- script looks at that column (called expectedFileExt) and then looks for 1 of each of those file extensions in the given folder
• Currently not erroring if file with incorrect extension is found, but this should be built in before Wednesday's DevOps

Other to-do items:

• Scripts should check whether the suffix found in file names matches the session number of the folder those files are in
• gentracker needs to be written so that it looks at the variable name + the allowedSuffix in the data dictionary and then uses that to look for the appropriate column in redcap files (variablename_suffix_complete) and create columns in the tracker accordingly
• Scripts need to be written to handle the allowedSuffix for pidentity and plang variables the same way it handles redcap data

[lillylaplace]

Summary from meeting 8/3:

David's to-do list:

• [x] Edit gentracker script so that it will create column names in the tracker that include the variable name + the allowed suffix from the data dictionary
• [x] Edit scripts regarding corrected.txt files --> if there is a corrected.txt file in a given folder, script should not error if there is a mismatch between file name and data dictionary but script should still error if there is a mismatch between file name and the folder it's in

My to-do list:

• [x] Update data dictionary to reflect changes (add expectedFileExt column, format pidentity and plang variables with allowedSuffix the same way we do for redcap_data) and push to instruments
• [x] Create tracker spreadsheet to be used for testing all possible errors

Other points from discussion:

• [x] We need to create a clear protocol around the corrected.txt files
• If an error in a file name is found, someone must (1) manually correct the error, (2) add a text file called "corrected.txt", then (3) manually move the corrected files to sourcedata/checked
• This protocol should only be followed in the case of a mismatch between file name and data dictionary; this protocol should NOT be followed in the case of a mismatch between file name and folder
• We will no longer use "no-digi", "no-eeg", etc. to denote that a folder will not contain the expected files; only using "corrected.txt" from here on out
• [x] In the future, we might want to address the fact that there could be errors in sourcedata/checked that we want the scripts to check for

[lillylaplace]

Notes from 8/15 meeting:

• we will be changing the output for pidentity and plang in the tracker so that for pidentity, 8 = primary and 9 = secondary and for plang, 1 = English and 2 = Spanish (involves change to allowedValues column in data dictionary as well as script changes)