QC w/ ngs-qc gen & plot

[kelly-sovacool]

old pipeliner code: https://github.com/CCBR/Pipeliner/blob/86c6ccaa3d58381a0ffd696bbf9c047e4f991f9e/Rules/InitialChIPseqQC.snakefile#L867-L921

[kelly-sovacool]

waiting on a docker container https://github.com/CCBR/Dockers/issues/11

[kelly-sovacool]

log from ngsqc_plot job in tutorial (/data/CCBR_Pipeliner/testdata/chipseq_tutorial_working_dir/slurmfiles/slurm-60089088.out)

[Mon Jun 22 00:48:44 2020] Building DAG of jobs...
[Mon Jun 22 00:48:44 2020] Using shell: /usr/bin/bash
[Mon Jun 22 00:48:44 2020] Provided cores: 4
[Mon Jun 22 00:48:44 2020] Rules claiming more threads will be scaled down.
[Mon Jun 22 00:48:44 2020] Job counts:
[Mon Jun 22 00:48:44 2020]  count   jobs
[Mon Jun 22 00:48:44 2020]  1   ngsqc_plot
[Mon Jun 22 00:48:44 2020]  1

[Mon Jun 22 00:48:44 2020] rule ngsqc_plot:
[Mon Jun 22 00:48:44 2020]     input: /scratch/kopardevn/chipseq_demo/QC/CTCF_ChIP_MEF_p20_1.Q5DD.NGSQC_report.txt, /scratch/kopardevn/chipseq_demo/QC/CTCF_ChIP_MEF_p20_2.Q5DD.NGSQC_report.txt, /scratch/kopardevn/chipseq_demo/QC/CTCF_ChIP_macrophage_p20_1.Q5DD.NGSQC_report.txt, /scratch/kopardevn/chipseq_demo/QC/CTCF_ChIP_macrophage_p20_2.Q5DD.NGSQC_report.txt, /scratch/kopardevn/chipseq_demo/QC/CTCF_ChIP_macrophage_p3_1.Q5DD.NGSQC_report.txt, /scratch/kopardevn/chipseq_demo/QC/CTCF_ChIP_macrophage_p3_2.Q5DD.NGSQC_report.txt, /scratch/kopardevn/chipseq_demo/QC/WCE_p20_input.Q5DD.NGSQC_report.txt, /scratch/kopardevn/chipseq_demo/QC/WCE_p3_input.Q5DD.NGSQC_report.txt
[Mon Jun 22 00:48:44 2020]     output: /scratch/kopardevn/chipseq_demo/QC/MEF_p20.NGSQC.Q5DD.png, /scratch/kopardevn/chipseq_demo/QC/Macrophage_p20.NGSQC.Q5DD.png, /scratch/kopardevn/chipseq_demo/QC/Macrophage_p3.NGSQC.Q5DD.png
[Mon Jun 22 00:48:44 2020]     jobid: 0

[Mon Jun 22 00:48:44 2020] if [ ! -e /lscratch/$SLURM_JOBID ]; then mkdir /lscratch/$SLURM_JOBID ;fi
[Mon Jun 22 00:48:44 2020] cd /lscratch/$SLURM_JOBID;mkdir MEF_p20; cp /scratch/kopardevn/chipseq_demo/QC/CTCF_ChIP_MEF_p20_1.Q5DD.NGSQC_report.txt MEF_p20; cp /scratch/kopardevn/chipseq_demo/QC/CTCF_ChIP_MEF_p20_2.Q5DD.NGSQC_report.txt MEF_p20; cp /scratch/kopardevn/chipseq_demo/QC/WCE_p20_input.Q5DD.NGSQC_report.txt MEF_p20; mkdir Macrophage_p20; cp /scratch/kopardevn/chipseq_demo/QC/CTCF_ChIP_macrophage_p20_1.Q5DD.NGSQC_report.txt Macrophage_p20; cp /scratch/kopardevn/chipseq_demo/QC/CTCF_ChIP_macrophage_p20_2.Q5DD.NGSQC_report.txt Macrophage_p20; cp /scratch/kopardevn/chipseq_demo/QC/WCE_p20_input.Q5DD.NGSQC_report.txt Macrophage_p20; mkdir Macrophage_p3; cp /scratch/kopardevn/chipseq_demo/QC/CTCF_ChIP_macrophage_p3_1.Q5DD.NGSQC_report.txt Macrophage_p3; cp /scratch/kopardevn/chipseq_demo/QC/CTCF_ChIP_macrophage_p3_2.Q5DD.NGSQC_report.txt Macrophage_p3; cp /scratch/kopardevn/chipseq_demo/QC/WCE_p3_input.Q5DD.NGSQC_report.txt Macrophage_p3; python /scratch/kopardevn/chipseq_demo/Scripts/ngsqc_plot.py -d 'MEF_p20' -e 'Q5DD' -g 'MEF_p20'; python /scratch/kopardevn/chipseq_demo/Scripts/ngsqc_plot.py -d 'Macrophage_p20' -e 'Q5DD' -g 'Macrophage_p20'; python /scratch/kopardevn/chipseq_demo/Scripts/ngsqc_plot.py -d 'Macrophage_p3' -e 'Q5DD' -g 'Macrophage_p3'; mv MEF_p20/MEF_p20.NGSQC.Q5DD.png /scratch/kopardevn/chipseq_demo/QC/MEF_p20.NGSQC.Q5DD.png; mv Macrophage_p20/Macrophage_p20.NGSQC.Q5DD.png /scratch/kopardevn/chipseq_demo/QC/Macrophage_p20.NGSQC.Q5DD.png; mv Macrophage_p3/Macrophage_p3.NGSQC.Q5DD.png /scratch/kopardevn/chipseq_demo/QC/Macrophage_p3.NGSQC.Q5DD.png; mv MEF_p20/CTCF_ChIP_MEF_p20_1.Q5DD.NGSQC.txt /scratch/kopardevn/chipseq_demo/QC; mv MEF_p20/CTCF_ChIP_MEF_p20_2.Q5DD.NGSQC.txt /scratch/kopardevn/chipseq_demo/QC; mv MEF_p20/WCE_p20_input.Q5DD.NGSQC.txt /scratch/kopardevn/chipseq_demo/QC; mv Macrophage_p20/CTCF_ChIP_macrophage_p20_1.Q5DD.NGSQC.txt /scratch/kopardevn/chipseq_demo/QC; mv Macrophage_p20/CTCF_ChIP_macrophage_p20_2.Q5DD.NGSQC.txt /scratch/kopardevn/chipseq_demo/QC; mv Macrophage_p3/CTCF_ChIP_macrophage_p3_1.Q5DD.NGSQC.txt /scratch/kopardevn/chipseq_demo/QC; mv Macrophage_p3/CTCF_ChIP_macrophage_p3_2.Q5DD.NGSQC.txt /scratch/kopardevn/chipseq_demo/QC; mv Macrophage_p3/WCE_p3_input.Q5DD.NGSQC.txt /scratch/kopardevn/chipseq_demo/QC; 
[Mon Jun 22 00:48:47 2020] Finished job 0.
[Mon Jun 22 00:48:47 2020] 1 of 1 steps (100%) done

python calls from above log:

python /scratch/kopardevn/chipseq_demo/Scripts/ngsqc_plot.py -d 'MEF_p20' -e 'Q5DD' -g 'MEF_p20';
python /scratch/kopardevn/chipseq_demo/Scripts/ngsqc_plot.py -d 'Macrophage_p20' -e 'Q5DD' -g 'Macrophage_p20';
python /scratch/kopardevn/chipseq_demo/Scripts/ngsqc_plot.py -d 'Macrophage_p3' -e 'Q5DD' -g 'Macrophage_p3';

example plots

[kopardev]

Use "nciccbr/ccbr_ngsqc_0.31:v1" ... ngsqc in PATH

[kelly-sovacool]

Getting a segfault error with process NGSQC_GEN

2259703 Segmentation fault      (core dumped) ngsqc -v SRR1822157_1.TagAlign.bed hg38.fa.sizes

also occurred when I tried running it interactively:

Reading the assembly file: hg38.fa.sizes..
Checking BED file format..
Screening for overlaps between the genome file and the input dataset..
Sample and intersect files for replicate #1..
Sampling 90% density:
Number of reads to sample: 28
Sampling 70% density:
Number of reads to sample: 22
Sampling 50% density:
Number of reads to sample: 16
Compute QC indicators..
.command.sh: line 2: 2262538 Segmentation fault      (core dumped) /data/CCBR_Pipeliner/db/PipeDB/ChIPseq/NGSQC_linux_x86_64 -v SRR1822157_1.TagAlign.bed hg38.fa.sizes

[kelly-sovacool]

This segfault may be because the test dataset is tiny. Will circle back to this after implementing the rest of the main pipeline.

[kelly-sovacool]

Still segfaulting on a mouse dataset with the mm10 genome.