search

igordot / sns

Analysis pipelines for genomic sequencing data
https://igordot.github.io/sns
MIT License
64 stars 26 forks source link

ERROR: the value for setting EXP-TARGETS-BED must be specified logs-sbatch/slurm-18641451.out: qc-target-reads-gatk.sh ERROR: BED DOES NOT EXIST #9

Closed ptoru001 closed 2 years ago

ptoru001 commented 2 years ago

Good morming Igor,

I have 7 samples 5 normal and two tumor.

The bed file has already bee added and is the same one as I used last time.

-rwx------ 1 torunp01 hackerlab 11302942 Jul 7 12:12 TruSeq_Exome_TargetedRegions_v1.2.bed

Yet I am getting this error:

ogs-sbatch/slurm-18641451.out: get-set-setting.sh ERROR: the value for setting EXP-TARGETS-BED must be specified logs-sbatch/slurm-18641451.out: qc-target-reads-gatk.sh ERROR: BED DOES NOT EXIST logs-sbatch/slurm-18641451.out: get-set-setting.sh ERROR: the value for setting EXP-TARGETS-BED must be specified logs-sbatch/slurm-18641451.out: bam-ra-rc-gatk.sh ERROR: BED DOES NOT EXIST logs-sbatch/slurm-18641451.out: wes.sh ERROR: SEGMENT bam-ra-rc-gatk DID NOT FINISH logs-sbatch/slurm-18641452.out: get-set-setting.sh ERROR: the value for setting EXP-TARGETS-BED must be specified

Is there another file I must add?

Thank you Pedro

igordot commented 2 years ago

Are you sure you are adding the BED file to the correct project directory?

Also, please check the entire log file. Just the error line is usually not enough.

yaacoo commented 2 years ago

I get exactly the same error, however I did not add any bed file because I'm working with WGS.

line 95 at: https://github.com/igordot/sns/blob/main/segments/qc-target-reads-gatk.sh#L95

throws an exit 1 if there is no bed file, meaning bed is not optional. I added a line that creates a bed file of all chromosome coordinates if there is no bed file present:

awk 'OFS="\t"{print $1, "0", $2}' ${ref_fastq}.fai > ${proj_dir}/wgs.bed Does this seem like a good solution?

What is the easiest way to jump back into a specific stage or the workflow without running alignment again?

igordot commented 2 years ago

Since it's the same workflow for WES and WGS, you can add a full-genome BED file (the whole genome is targeted). There is a genome.bed in the reference directory.

You can run the pipeline again and it will skip the steps that successfully finished.

yaacoo commented 2 years ago

Yes that's what I figured. Thanks!

ptoru001 commented 2 years ago

Thanks Igor,

all samples ran fine after after i deleted and reuploaded the bed file provided by GTC...

Are any of the files that are generated through WES route also good to use for copy number in gistic?

if not, what would be a way to obtain these .seg files?

Thanks for your time Igor.

-pedro

igordot commented 2 years ago

There is a wes-pairs-cnv route for copy number analysis using Control-FREEC. It does not generate SEG files, but you may be able to generate them from other output that it generates.

ptoru001 commented 2 years ago

Thanks Igor,

I manually populated the fields in the pairs.csv file like this: ptm8,ptm_3_214887 ptm8,ptm_4_215138 ptm8,ptm_5_213291 ptm8,ptm_6_215107 ptm8,ptm_7_215865

then I dbl checked that the bed file is currently in the settings.txt as well as in the directory itself and i get this error: wes-pairs-snv.sh ERROR: WRONG NUMBER OF ARGUMENTS SUPPLIED logs-sbatch/slurm-18718525.out: wes-pairs-snv.sh ERROR: WRONG NUMBER OF ARGUMENTS SUPPLIED logs-sbatch/slurm-18718526.out: wes-pairs-snv.sh ERROR: WRONG NUMBER OF ARGUMENTS SUPPLIED logs-sbatch/slurm-18718527.out: wes-pairs-snv.sh ERROR: WRONG NUMBER OF ARGUMENTS SUPPLIED logs-sbatch/slurm-18718528.out: wes-pairs-snv.sh ERROR: WRONG NUMBER OF ARGUMENTS SUPPLIED logs-sbatch/slurm-18718529.out: wes-pairs-snv.sh ERROR: BAM DOES NOT EXIST logs-sbatch/slurm-18718530.out: wes-pairs-snv.sh ERROR: BAM DOES NOT EXIST logs-sbatch/slurm-18718532.out: wes-pairs-cnv.sh ERROR: WRONG NUMBER OF ARGUMENTS SUPPLIED logs-sbatch/slurm-18718533.out: wes-pairs-cnv.sh ERROR: WRONG NUMBER OF ARGUMENTS SUPPLIED logs-sbatch/slurm-18718534.out: wes-pairs-cnv.sh ERROR: WRONG NUMBER OF ARGUMENTS SUPPLIED logs-sbatch/slurm-18718535.out: wes-pairs-cnv.sh ERROR: WRONG NUMBER OF ARGUMENTS SUPPLIED logs-sbatch/slurm-18718536.out: wes-pairs-cnv.sh ERROR: WRONG NUMBER OF ARGUMENTS SUPPLIED logs-sbatch/slurm-18718537.out: wes-pairs-cnv.sh ERROR: BAM DOES NOT EXIST logs-sbatch/slurm-18718538.out: wes-pairs-cnv.sh ERROR: BAM DOES NOT EXIST logs-sbatch/slurm-18718543.out: get-set-setting.sh ERROR: the value for setting EXP-PROBES-BED must be specified logs-sbatch/slurm-18718543.out: cnvs-wes-freec.sh ERROR: BED DOES NOT EXIST logs-sbatch/slurm-18718545.out: get-set-setting.sh ERROR: the value for setting EXP-PROBES-BED must be specified logs-sbatch/slurm-18718545.out: cnvs-wes-freec.sh ERROR: BED DOES NOT EXIST

how come the pipeline is not running?

Thank you Pedro

igordot commented 2 years ago

There is probably a problem with file formatting (when you saved the file).

As I mentioned previously, if you see any errors, please check the full log file. The error line alone is not sufficient.

ptoru001 commented 2 years ago

thanks for your feedback...this is the error message from logs-sbatch

although ive deleted and readded the settings for hg19, deleted and added the bed file to the settings path.

I am still getting the same errors.

apologies for the trouble and thanks for your feedback.

========== ROUTE: wes-pairs-cnv ==========

CMD: bash /gpfs/data/hackerlab/2022-08-24/sns/segments/cnvs-wes-freec.sh /gpfs/data/hackerlab/2022-08-24 ptm8 /gpfs/data/hackerlab/2022-08-24/BAM-GATK-RA-RC/ptm8.dd.ra.rc.bam ptm_5_213291 /gpfs/data/hackerlab/2022-08-24/BAM-GATK-RA-RC/ptm_5_213291.dd.ra.rc.bam

========== SEGMENT: cnvs-wes-freec ==========

get-set-setting.sh ERROR: the value for setting GENOME-DIR must be specified

cnvs-wes-freec.sh ERROR: GENOME DIR DOES NOT EXIST

Fri Sep 2 12:00:02 EDT 2022

igordot commented 2 years ago

I don't understand. You say you are getting the same error, but posting a new error.

ptoru001 commented 2 years ago

Good Morning,

This is from printing errors from slurm...i get no bed file error and no genome directory found.

These are the errors from printing logs-sbatch:

get-set-setting.sh ERROR: the value for setting GENOME-DIR must be specified cnvs-wes-freec.sh ERROR: GENOME DIR DOES NOT EXIST get-set-setting.sh ERROR: the value for setting EXP-PROBES-BED must be specified cnvs-wes-freec.sh ERROR: BED DOES NOT EXIST The bed file has been deleted and added but still gives errors above.

igordot commented 2 years ago

I understand where the errors are from. I meant that this is not an error you reported earlier. You introduced a new one.

When you edited settings.txt, you may have introduced non-UNIX line endings. You can check for yourself with cat -A settings.txt. You should see just $, not ^M$, at the end of every line.

To fix, run:

module load dos2unix/7.4.0
dos2unix settings.txt
ptoru001 commented 2 years ago

hello igor,

this is what the settings.txt looks like GENOME-DIR|/gpfs/data/igorlab/ref/hg19 REF-FASTA|/gpfs/data/igorlab/ref/hg19/genome.fa REF-BWA|/gpfs/data/igorlab/ref/hg19/BWA/genome.fa REF-DICT|/gpfs/data/igorlab/ref/hg19/genome.dict EXP-TARGETS-BED|/gpfs/data/hackerlab/2022_08/TruSeq_Exome_TargetedRegions_v1.2.bed EXP-PROBES-BED|/gpfs/data/hackerlab/2022_08/TruSeq_Exome_TargetedRegions_v1.2.bed REF-CHROMSIZES|/gpfs/data/igorlab/ref/hg19/chrom.sizes

this is the error i am getting:

Control-FREEC v11.6 : a method for automatic detection of copy number alterations, subclones and for accurate estimation of contamination and main ploidy using deep-sequencing data Multi-threading mode using 4 threads ..consider the sample being male ..Breakpoint threshold for segmentation of copy number profiles is 1.2 ..telocenromeric set to 50000 ..FREEC is not going to adjust profiles for a possible contamination by normal cells ..Window = 0 was set ..Output directory: . ..Directory with files containing chromosome sequences: /gpfs/data/igorlab/ref/iGenomes/Homo_sapiens/UCSC/hg19/Sequence/Chromosomes/ ..will use a threshold of 10 read(s) per SNP position to calculate beta allel frequency (BAF) values ..Sample file: /gpfs/data/hackerlab/2022-08-24/BAM-GATK-RA-RC/ptm8.dd.ra.rc.bam ..Sample input format: BAM ..will use this instance of samtools: 'samtools' to read BAM files ..Control file: /gpfs/data/hackerlab/2022-08-24/BAM-GATK-RA-RC/ptm_7_215865.dd.ra.rc.bam ..Input format for the control file: BAM FREEC will create a pileup to compute BAF profile! ...File with SNPs : /gpfs/data/hackerlab/2022-08-24/logs-cnvs-wes-freec/ptm8-ptm_7_215865/snps.bed ..forceGCcontentNormalization was set to 1: will use GC-content to normalize the read count data ..minimal expected GC-content (general parameter "minExpectedGC") was set to 0.35 ..maximal expected GC-content (general parameter "maxExpectedGC") was set to 0.55 ..Minimal CNA length (in windows) is 5 ..File with chromosome lengths: /gpfs/data/hackerlab/2022-08-24/logs-cnvs-wes-freec/ptm8-ptm_7_215865/chrs.len.txt ..File /gpfs/data/hackerlab/2022-08-24/logs-cnvs-wes-freec/ptm8-ptm_7_215865/chrs.len.txt was read ..Using the default minimal mappability value of 0.85 ..Mappability file/gpfs/data/igorlab/ref/hg19/FREEC/out100m2_hg19.gem be used: all low mappability positions will be discarded ..uniqueMatch = FALSE ..average ploidy set to 2 ..break-point type set to 4 ..noisyData set to 1 ..minimal number of reads per window in the control sample is set to 25 ..Control-FREEC will not look for subclones Creating Pileup file to compute BAF profile... ..will increase flanking regions by 107 bp [mpileup] 1 samples in 1 input files

the CNV-FREEC folder is empty

thank you

igordot commented 2 years ago

I don't understand. There is no error.