data-monitoring: bugs/features before release

[jessb0t]

hallMonitor.sub

1. I re-ran hallMonitor after deleting a Pavlovia file in “raw.” The script caught it, but the error content seems weird to me. It says “Improper file name , does not meet standard” when, in fact, it is a deletion from “raw” for data that already exists in “checked.” So the “improper file name” is the total absence of a filename. Should we change the wording on the error, or is this intentional (in which case, please edify me!) :point_right: correct error for empty directories
2. I tested creating a mismatch between the subject ID within a Pavlovia file and the subject ID on the folder name. This did not throw an error. I believe we discussed this at some point (way way back), but I’m not sure that we actually put that into the script specifications when you started working on it. (You can check it out at /home/data/NDClab/datasets/test-readAloud-v3/sourcedata/raw/pavlovia/sub-150153 where the .csv file actually has 150154 instead. :point_right: investigate, but don't put too much effort into fixing for now
3. As you know, we have several instruments that are two-in-one, like SIAS6SPS6. For some studies (like Valence), we launched data collection before we decided to consistently use the double questionnaire name in all instrument items. So instruments script needs these:

   sias6sps6_b_i1  sias6sps6_b_i2  sias6sps6_b_i3  sias6sps6_b_i4  sias6sps6_b_i5  sias6sps6_b_i6  sias6sps6_b_i1  sias6sps6_b_i2  sias6sps6_b_i3  sias6sps6_b_i4  sias6sps6_b_i5  sias6sps6_b_i6

   …but it gets these:

   sias6_b_i1  sias6_b_i2  sias6_b_i3  sias6_b_i4  sias6_b_i5  sias6_b_i6  sps6_b_i1   sps6_b_i2   sps6_b_i3   sps6_b_i4   sps6_b_i5   sps6_b_i6

   We have corrected this for future data collection, but George was wondering if we could build this header renaming into hallMonitor, since it only affects a very limited number of instruments? That is: when hallMonitor is checking the REDCap file, it checks for the “bad” variable names and, if found, replaces them with the “good” variable names? :point_right: Farukh add to hallMonitor.sub for Valence for sias6sps6 example above (in bash)

preprocess.sub

1. Is it possible to automate the filepaths somehow so that the user does not need to manually modify preprocess.sub once it is set up, and it would automatically capture the newest REDCap file in /sourcedata/checked? :point_right: Farukh to update to most recent REDCap file from sourcedata/checked
2. We can remove "psychopy-preproc.R" from preprocess.sub (older version of script, now replaced by (preprocPavlovia.R"). :point_right: Remove.
3. Why do we run one of the R scripts in one section and two in another? That is, why not put "preprocPavlovia.R" into the same container as the two that require special packages? :point_right: Move to same container as two scripts with packages.
4. When I tested on test-readAloud-v3, only the instruments portion successfully ran (that is, there was no output for the three R scripts). :point_right: Adjust so that preprocess.sub reads R in code folder

inst-tracker.py

1. Can we add comments to this code so that instructions to a new researcher for setup are clear? :point_right: Farukh to add some instructional comments
2. The three of us should review the possible values and see if we want to reduce to 11/19 or expand the code to capture all possible values. :soon: :soon: :soon:
3. Is PR/branch merge ready in instruments? (Unclear how this is working on HPC if that isn't yet merged to main branch.) :no_entry_sign: Hold for now. Tracks all subscores not just those specified by researcher.

wiki

Discuss slurm script customization. :soon: :soon: :soon:

[jessb0t]

Jess Testing Notes

• check changing a raw Pavlovia file (not just deleting)
• check hallMonitor behavior if delete raw file and re-run
• check zoom filename and folder name match