Allow replicates when ingesting genome-wide CRISPR screens

[DSuveges]

So far, when ingesting CRISPR knock-out screens for PPP, we have one screen for each study. However OTAR2-061 introduces the concept of "replicates". These are different experimental runs of the same study. To increase confidence in the findings, we potentially need to read data from multiple replicates and only keep genes as hits that were identified as significant in all the replicates.

Tasks

• [x] Update PPP configuration and config schema.
• [x] Update config files on gs://otar013-ppp/PPP-evidence-configuration
• [x] Upload input files to bucket: gs://otar013-ppp/ot_crispr/input_data/OTAR2061
• [x] Make parser aware of replicate identifier.
• [x] Generate evidence
• [x] Validate

[DSuveges]

Uploading mageck files to evidence bucket:

FILE_LIST=(
    WT_IFNg_D14_exp1_vs_JAK1_KO_C1_IFNg_D14_exp1.gene_summary.txt   WT_CTRL_D14_exp1_vs_JAK1_KO_C1_CTRL_D14_exp1.gene_summary.txt
    WT_IFNg_D14_exp1_vs_JAK1_KO_C3_IFNg_D14_exp1.gene_summary.txt   WT_CTRL_D14_exp1_vs_JAK1_KO_C3_CTRL_D14_exp1.gene_summary.txt
    WT_IFNg_D14_exp2_vs_JAK1_KO_C1_IFNg_D14_exp2.gene_summary.txt   WT_CTRL_D14_exp2_vs_JAK1_KO_C1_CTRL_D14_exp2.gene_summary.txt
    WT_IFNg_D14_exp2_vs_JAK1_KO_C3_IFNg_D14_exp2.gene_summary.txt   WT_CTRL_D14_exp2_vs_JAK1_KO_C3_CTRL_D14_exp2.gene_summary.txt
)

DATA_FOLDER=/Users/dsuveges/project_data/OTAR2061
TARGET_BUCKET=gs://otar013-ppp/ot_crispr/input_data/OTAR2061

grep -f <(for f in ${files[@]}; do echo $f; done) \
    <(find ${DATA_FOLDER} -type f) \
    | while read f; do 
        gsutil cp "${f}" ${TARGET_BUCKET}/
done

[DSuveges]

• [x] Datafiles for all 16 studies from OTAR2061 were added to bucket.
• [x] New PPP configuration was generated based on the updated table.
• [x] PPP configuration was uploaded to google bucket.
• [x] All evidence successfully validated against schema.