search

kundajelab / chrombpnet

Bias factorized, base-resolution deep learning models of chromatin accessibility (chromBPNet)
https://github.com/kundajelab/chrombpnet/wiki
MIT License
121 stars 34 forks source link

Blocking issue with Step 2 of Tutorial (bam_to_bigwig.sh) #51

Closed SuhasSrinivasan closed 2 years ago

SuhasSrinivasan commented 2 years ago

There is an issue Error: Can't open genome filehg38.chrom.sizesExiting... Originating from: `https://github.com/kundajelab/chrombpnet/blob/135f548ab0f4c3869a40442111b3c09d78bb7a99/src/helpers/preprocessing/bam_to_bigwig.sh#L52

Full printed message

(base) root@8dd36de3faec:/scratch/chrombpnet# bash step2_make_bigwigs_from_bams.sh data/downloads/merged.bam data/downloads/K562 ATAC_PE data/downloads/hg38.fa hg38.chrom.sizes
2022-06-01_17-40-07: shift ATAC PE data
2022-06-01_17-40-07: samtools view -b -@50 data/downloads/merged.bam | bedtools bamtobed -i stdin | awk -v OFS="\t" \'{if ($6=="-"){print $1,$2,$3-4,$4,$5,$6} else if ($6=="+") {print $1,$2+4,$3,$4,$5,$6}}\' | bedtools genomecov -bg -5 -i stdin -g  hg38.chrom.sizes | bedtools sort -i stdin > data/downloads/K562.tmp2
Error: Can't open genome filehg38.chrom.sizesExiting...
2022-06-01_17-40-11: bedGraphToBigWig data/downloads/K562.tmp2 hg38.chrom.sizes data/downloads/K562\_unstranded.bw
Couldn't open hg38.chrom.sizes
"bedGraphToBigWig $output_prefix".tmp2" $chrom_sizes $output_prefix"_unstranded.bw"" command filed with exit code 255.
"bash $PWD/src/helpers/preprocessing/bam_to_bigwig.sh $in_bam $bigwig_prefix $data_type $chrom_sizes $logfile" command filed with exit code 255.

Troubleshooting tried:

  1. Same issue with Conda setup
  2. Same issue with Docker setup
  3. Same issue after redownloading file
  4. Verified through cat hg38.chrom.sizes file is readable and not corrupted
annashcherbina commented 2 years ago

Hi @SuhasSrinivasan , thank you for reporting these issues. Just a heads up, a lot of the problems should go away shortly once we release a pipy module for installation. You can get a head start on this by cloning the following branch locally:

https://github.com/kundajelab/chrombpnet/tree/create-pip-module

and then installing with pip -e on your system: 

pip install -e chrombpnet
SuhasSrinivasan commented 2 years ago

Hi @annashcherbina, I believe this issue does not fall under the Installation category. bam_to_bigwig.sh is failing to read to the file: hg38.chrom.sizes

annashcherbina commented 2 years ago

so it actually does because of how the scripts are linked. See here: https://github.com/kundajelab/chrombpnet/blob/create-pip-module/setup.py#L37

It was a linking issue in the original branch, fixed in this one.

Please see example of how to execute the full workflow and avoid this issue:

https://github.com/kundajelab/chrombpnet/blob/create-pip-module/tests/full_workflow.sh

panushri25 commented 2 years ago

Can you change this to

bash step2_make_bigwigs_from_bams.sh data/downloads/merged.bam data/downloads/K562 ATAC_PE data/downloads/hg38.fa data/downloads/hg38.chrom.sizes

panushri25 commented 2 years ago

your hg38.chrom.sizes is in data/downloads/hg38.chrom.sizes correct?

panushri25 commented 2 years ago

@SuhasSrinivasan Can you confirm that this is working after this change for you?

panushri25 commented 2 years ago

I got the confirmation from another user accessing this. I am going to close this for now. Please feel free to open this if not resolved! Thank you @SuhasSrinivasan