Cannot generate the wigToBigWig file on large genome

[sebel76]

Hi Micheal,

I ran the analysis another time yesterday. I can go through the whole analysis and return miRNA annotation files and figures. I ran the analysis in two step; mapping sRNAs and annotating miRNA.

I observed one bug about the bigwig file. It work well when analysing small genomes <3.5Gb like Brachypodium sp. or Raddia sp. but it failed on larger genomes. Bellow, I extract the log for one of the species:

Creating browser tracks by readgroup using ShortTracks ShortTracks version 1.0

Required executable samtools : /Users/sbelanger/mambaforge/envs/snakemake/bin/samtools Required executable wigToBigWig : /Users/sbelanger/mambaforge/envs/snakemake/bin/wigToBigWig

Getting raw depths samtools view -F 256 -r evo_RS_SB16 -h srna/Tae/merged_alignments.bam | samtools depth - > srna/Tae/merged_alignments_evo_RS_SB16_depths.txt samtools view -F 256 -r evo_RS_SB19 -h srna/Tae/merged_alignments.bam | samtools depth - > srna/Tae/merged_alignments_evo_RS_SB19_depths.txt samtools view -F 256 -r evo_RS_SB20 -h srna/Tae/merged_alignments.bam | samtools depth - > srna/Tae/merged_alignments_evo_RS_SB20_depths.txt samtools view -F 256 -r evo_RS_SB21 -h srna/Tae/merged_alignments.bam | samtools depth - > srna/Tae/merged_alignments_evo_RS_SB21_depths.txt samtools view -F 256 -r evo_RS_SB22 -h srna/Tae/merged_alignments.bam | samtools depth - > srna/Tae/merged_alignments_evo_RS_SB22_depths.txt samtools view -F 256 -r evo_RS_SB24 -h srna/Tae/merged_alignments.bam | samtools depth - > srna/Tae/merged_alignments_evo_RS_SB24_depths.txt samtools view -F 256 -r evo_RS_SB26 -h srna/Tae/merged_alignments.bam | samtools depth - > srna/Tae/merged_alignments_evo_RS_SB26_depths.txt samtools view -F 256 -r evo_RS_SB28 -h srna/Tae/merged_alignments.bam | samtools depth - > srna/Tae/merged_alignments_evo_RS_SB28_depths.txt samtools view -F 256 -r evo_RS_SB29 -h srna/Tae/merged_alignments.bam | samtools depth - > srna/Tae/merged_alignments_evo_RS_SB29_depths.txt

Writing wiggle files srna/Tae/merged_alignments_evo_RS_SB16.wig srna/Tae/merged_alignments_evo_RS_SB19.wig srna/Tae/merged_alignments_evo_RS_SB20.wig srna/Tae/merged_alignments_evo_RS_SB21.wig srna/Tae/merged_alignments_evo_RS_SB22.wig srna/Tae/merged_alignments_evo_RS_SB24.wig srna/Tae/merged_alignments_evo_RS_SB26.wig srna/Tae/merged_alignments_evo_RS_SB28.wig srna/Tae/merged_alignments_evo_RS_SB29.wig

Writing bigwig files wigToBigWig srna/Tae/merged_alignments_evo_RS_SB16.wig chromSizes.txt srna/Tae/merged_alignments_evo_RS_SB16.bw hashMustFindVal: 'Tae_1A' not found wigToBigWig srna/Tae/merged_alignments_evo_RS_SB19.wig chromSizes.txt srna/Tae/merged_alignments_evo_RS_SB19.bw hashMustFindVal: 'Tae_1A' not found wigToBigWig srna/Tae/merged_alignments_evo_RS_SB20.wig chromSizes.txt srna/Tae/merged_alignments_evo_RS_SB20.bw hashMustFindVal: 'Tae_1A' not found wigToBigWig srna/Tae/merged_alignments_evo_RS_SB21.wig chromSizes.txt srna/Tae/merged_alignments_evo_RS_SB21.bw hashMustFindVal: 'Tae_1A' not found wigToBigWig srna/Tae/merged_alignments_evo_RS_SB22.wig chromSizes.txt srna/Tae/merged_alignments_evo_RS_SB22.bw hashMustFindVal: 'Tae_1A' not found wigToBigWig srna/Tae/merged_alignments_evo_RS_SB24.wig chromSizes.txt srna/Tae/merged_alignments_evo_RS_SB24.bw hashMustFindVal: 'Tae_1A' not found wigToBigWig srna/Tae/merged_alignments_evo_RS_SB26.wig chromSizes.txt srna/Tae/merged_alignments_evo_RS_SB26.bw hashMustFindVal: 'Tae_1A' not found wigToBigWig srna/Tae/merged_alignments_evo_RS_SB28.wig chromSizes.txt srna/Tae/merged_alignments_evo_RS_SB28.bw hashMustFindVal: 'Tae_1A' not found wigToBigWig srna/Tae/merged_alignments_evo_RS_SB29.wig chromSizes.txt srna/Tae/merged_alignments_evo_RS_SB29.bw hashMustFindVal: 'Tae_1A' not found

Getting raw depths

Preparing commands, counting overall reads

samtools view -F 272 -e "qlen==21" -h srna/Tae/merged_alignments.bam | samtools depth - > srna/Tae/merged_alignments_21_p_depths.txt samtools view -F 256 -f 16 -e "qlen==21" -h srna/Tae/merged_alignments.bam | samtools depth - > srna/Tae/merged_alignments_21_m_depths.txt samtools view -F 272 -e "qlen==22" -h srna/Tae/merged_alignments.bam | samtools depth - > srna/Tae/merged_alignments_22_p_depths.txt samtools view -F 256 -f 16 -e "qlen==22" -h srna/Tae/merged_alignments.bam | samtools depth - > srna/Tae/merged_alignments_22_m_depths.txt samtools view -F 272 -e "qlen==23 || qlen==24" -h srna/Tae/merged_alignments.bam | samtools depth - > srna/Tae/merged_alignments_23-24_p_depths.txt samtools view -F 256 -f 16 -e "qlen==23 || qlen==24" -h srna/Tae/merged_alignments.bam | samtools depth - > srna/Tae/merged_alignments_23-24_m_depths.txt samtools view -F 272 -e "qlen < 21 || qlen > 24" -h srna/Tae/merged_alignments.bam | samtools depth - > srna/Tae/merged_alignments_other_p_depths.txt samtools view -F 256 -f 16 -e "qlen < 21 || qlen > 24" -h srna/Tae/merged_alignments.bam | samtools depth - > srna/Tae/merged_alignments_other_m_depths.txt

Writing wiggle files srna/Tae/merged_alignments_21_p.wig srna/Tae/merged_alignments_21_m.wig srna/Tae/merged_alignments_22_m.wig srna/Tae/merged_alignments_23-24_p.wig srna/Tae/merged_alignments_23-24_m.wig srna/Tae/merged_alignments_other_p.wig srna/Tae/merged_alignments_other_m.wig

Writing bigwig files wigToBigWig srna/Tae/merged_alignments_21_p.wig chromSizes.txt srna/Tae/merged_alignments_21_p.bw Couldn't open chromSizes.txt wigToBigWig srna/Tae/merged_alignments_21_m.wig chromSizes.txt srna/Tae/merged_alignments_21_m.bw Couldn't open chromSizes.txt wigToBigWig srna/Tae/merged_alignments_22_p.wig chromSizes.txt srna/Tae/merged_alignments_22_p.bw Couldn't open chromSizes.txt wigToBigWig srna/Tae/merged_alignments_22_m.wig chromSizes.txt srna/Tae/merged_alignments_22_m.bw Couldn't open chromSizes.txt wigToBigWig srna/Tae/merged_alignments_23-24_p.wig chromSizes.txt srna/Tae/merged_alignments_23-24_p.bw Couldn't open chromSizes.txt wigToBigWig srna/Tae/merged_alignments_23-24_m.wig chromSizes.txt srna/Tae/merged_alignments_23-24_m.bw Couldn't open chromSizes.txt wigToBigWig srna/Tae/merged_alignments_other_p.wig chromSizes.txt srna/Tae/merged_alignments_other_p.bw Couldn't open chromSizes.txt wigToBigWig srna/Tae/merged_alignments_other_m.wig chromSizes.txt srna/Tae/merged_alignments_other_m.bw Couldn't open chromSizes.txt

Best, Sébastien

[MikeAxtell]

Thanks Sébastien,

Perhaps this is related to the other big genomes issues you raised in #118 ? I will look into it when I work on testing out on giant genomes.

[sebel76]

Thank you, Mike! Let me know if you find it, I will enjoy to learn and re-process my files.

[MikeAxtell]

This was an issue in the helper script ShortTracks that ShortStack uses to make genome browser files. The issue was the non-recognition of the .csi style of BAM indices that must be used when there are one or more chromosomes that are longer than about 500Mb. The new ShortTracks release (v1.1) will solve this.