NDCLab / lab-devOps

NDCLab mgmt and operations
GNU Affero General Public License v3.0
0 stars 0 forks source link

data-preprocessing: REDCap script #183

Closed jessb0t closed 2 years ago

jessb0t commented 2 years ago

Pulls input from: checked Outputs to: preprocessed

Script checks for any newer version of the REDCap export .csv file. If found, runs that file through the instruments scoring script.

F-said commented 2 years ago

Some questions @jessb0t

jessb0t commented 2 years ago

@F-said Yes, the CSVs do have a standardized name. It begins with a truncated version of the REDCap project (and we make new REDCap projects for the same study when we make major modifications to the questionnaires that participants answer). For instance, here is Social Context: image.png The standardized naming (direct from REDCap) also includes the export date. I do not know what the four number at the end represent.

Yes, older files will persist (see screenshot above). For now, I believe we have decided to live with all the duplication in order to avoid any risk of losing data. But would love to brainstorm future solutions that would make this all "cleaner."

No, output CSVs should be placed in the derivatives/preprocessed folder. In effect, we are running the "instruments" script on the checked files of a particular study/folder structure and outputting the results to the preprocessed folder dedicated to that study.