BDI-pathogens / phyloscanner

Phylogenetics between and within hosts at once, all along the genome.
GNU General Public License v3.0
44 stars 14 forks source link

Bam files have no reads #70

Closed Daniele-Pantano closed 1 year ago

Daniele-Pantano commented 1 year ago

Good morning, I am trying to run an analysis with SARS-CoV-2 samples taken within the same households and I decided to experiment a little with phyloscanner (I am a beginner in using Linux and programming in general). I set up the system with all the repositories and libraries needed within my environment in anaconda (I set up python 2 as suggested) and started the analysis. At first I got a problem with Mafft (process killed) but I solved it using the advice from a previous issues (#25) but, unfortunately, I run into this and I am not sure why...

(py2) user@user-VirtualBox:~/phyloscanner$ ~/phyloscanner/phyloscanner_make_trees.py BamAndRefListHH001.csv --auto-window-params 1000,0 --alignment-of-other-refs ref_wuhan.fasta --pairwise-align-to MN908947.3 --x-mafft2 mafft

phyloscanner was called thus: /home/user/phyloscanner/phyloscanner_make_trees.py BamAndRefListHH001.csv --auto-window-params 1000,0 --alignment-of-other-refs ref_wuhan.fasta --pairwise-align-to MN908947.3 --x-mafft2 mafft Warning: RAxML files are present in the working directory. If their names clash with those that phyloscanner will try to create, RAxML will fail to run. Continuing. Now extracting and processing reads in window 1-1000 Warning: bam file 20430884332_refined_RM.bam has no reads (after processing) that fully span the window 1-1000. Warning: bam file 20430884632_refined_RM.bam has no reads (after processing) that fully span the window 1-1000. Warning: bam file 20430884732_refined_RM.bam has no reads (after processing) that fully span the window 1-1000. Warning: bam file 20440569932_refined_RM.bam has no reads (after processing) that fully span the window 1-1000. WARNING: no bam file had any reads in the window 1-1000. Skipping to the next window. Now extracting and processing reads in window 1001-2000 Warning: bam file 20430884332_refined_RM.bam has no reads (after processing) that fully span the window 1001-2000. Warning: bam file 20430884632_refined_RM.bam has no reads (after processing) that fully span the window 1001-2000. Warning: bam file 20430884732_refined_RM.bam has no reads (after processing) that fully span the window 1001-2000. Warning: bam file 20440569932_refined_RM.bam has no reads (after processing) that fully span the window 1001-2000. WARNING: no bam file had any reads in the window 1001-2000. Skipping to the next window. Now extracting and processing reads in window 2001-3000 Warning: bam file 20430884332_refined_RM.bam has no reads (after processing) that fully span the window 2001-3000. Warning: bam file 20430884632_refined_RM.bam has no reads (after processing) that fully span the window 2001-3000. Warning: bam file 20430884732_refined_RM.bam has no reads (after processing) that fully span the window 2001-3000. Warning: bam file 20440569932_refined_RM.bam has no reads (after processing) that fully span the window 2001-3000. WARNING: no bam file had any reads in the window 2001-3000. Skipping to the next window. Now extracting and processing reads in window 3001-4000 Warning: bam file 20430884332_refined_RM.bam has no reads (after processing) that fully span the window 3001-4000. Warning: bam file 20430884632_refined_RM.bam has no reads (after processing) that fully span the window 3001-4000. Warning: bam file 20430884732_refined_RM.bam has no reads (after processing) that fully span the window 3001-4000. Warning: bam file 20440569932_refined_RM.bam has no reads (after processing) that fully span the window 3001-4000. WARNING: no bam file had any reads in the window 3001-4000. Skipping to the next window. Now extracting and processing reads in window 4001-5000 Warning: bam file 20430884332_refined_RM.bam has no reads (after processing) that fully span the window 4001-5000. Warning: bam file 20430884632_refined_RM.bam has no reads (after processing) that fully span the window 4001-5000. Warning: bam file 20430884732_refined_RM.bam has no reads (after processing) that fully span the window 4001-5000. Warning: bam file 20440569932_refined_RM.bam has no reads (after processing) that fully span the window 4001-5000. WARNING: no bam file had any reads in the window 4001-5000. Skipping to the next window. Now extracting and processing reads in window 5001-6000 Warning: bam file 20430884332_refined_RM.bam has no reads (after processing) that fully span the window 5001-6000. Warning: bam file 20430884632_refined_RM.bam has no reads (after processing) that fully span the window 5001-6000. Warning: bam file 20430884732_refined_RM.bam has no reads (after processing) that fully span the window 5001-6000. Warning: bam file 20440569932_refined_RM.bam has no reads (after processing) that fully span the window 5001-6000. WARNING: no bam file had any reads in the window 5001-6000. Skipping to the next window. Now extracting and processing reads in window 6001-7000 Warning: bam file 20430884332_refined_RM.bam has no reads (after processing) that fully span the window 6001-7000. Warning: bam file 20430884632_refined_RM.bam has no reads (after processing) that fully span the window 6001-7000. Warning: bam file 20430884732_refined_RM.bam has no reads (after processing) that fully span the window 6001-7000. Warning: bam file 20440569932_refined_RM.bam has no reads (after processing) that fully span the window 6001-7000. WARNING: no bam file had any reads in the window 6001-7000. Skipping to the next window. Now extracting and processing reads in window 7001-8000 Warning: bam file 20430884332_refined_RM.bam has no reads (after processing) that fully span the window 7001-8000. Warning: bam file 20430884632_refined_RM.bam has no reads (after processing) that fully span the window 7001-8000. Warning: bam file 20430884732_refined_RM.bam has no reads (after processing) that fully span the window 7001-8000. Warning: bam file 20440569932_refined_RM.bam has no reads (after processing) that fully span the window 7001-8000. WARNING: no bam file had any reads in the window 7001-8000. Skipping to the next window. Now extracting and processing reads in window 8001-9000 Warning: bam file 20430884332_refined_RM.bam has no reads (after processing) that fully span the window 8001-9000. Warning: bam file 20430884632_refined_RM.bam has no reads (after processing) that fully span the window 8001-9000. Warning: bam file 20430884732_refined_RM.bam has no reads (after processing) that fully span the window 8001-9000. Warning: bam file 20440569932_refined_RM.bam has no reads (after processing) that fully span the window 8001-9000. WARNING: no bam file had any reads in the window 8001-9000. Skipping to the next window. Now extracting and processing reads in window 9001-10000 Warning: bam file 20430884332_refined_RM.bam has no reads (after processing) that fully span the window 9001-10000. Warning: bam file 20430884632_refined_RM.bam has no reads (after processing) that fully span the window 9001-10000. Warning: bam file 20430884732_refined_RM.bam has no reads (after processing) that fully span the window 9001-10000. Warning: bam file 20440569932_refined_RM.bam has no reads (after processing) that fully span the window 9001-10000. WARNING: no bam file had any reads in the window 9001-10000. Skipping to the next window. Now extracting and processing reads in window 10001-11000 Warning: bam file 20430884332_refined_RM.bam has no reads (after processing) that fully span the window 10001-11000. Warning: bam file 20430884632_refined_RM.bam has no reads (after processing) that fully span the window 10001-11000. Warning: bam file 20430884732_refined_RM.bam has no reads (after processing) that fully span the window 10001-11000. Warning: bam file 20440569932_refined_RM.bam has no reads (after processing) that fully span the window 10001-11000. WARNING: no bam file had any reads in the window 10001-11000. Skipping to the next window. Now extracting and processing reads in window 11001-12000 Warning: bam file 20430884332_refined_RM.bam has no reads (after processing) that fully span the window 11001-12000. Warning: bam file 20430884632_refined_RM.bam has no reads (after processing) that fully span the window 11001-12000. Warning: bam file 20430884732_refined_RM.bam has no reads (after processing) that fully span the window 11001-12000. Warning: bam file 20440569932_refined_RM.bam has no reads (after processing) that fully span the window 11001-12000. WARNING: no bam file had any reads in the window 11001-12000. Skipping to the next window. Now extracting and processing reads in window 12001-13000 Warning: bam file 20430884332_refined_RM.bam has no reads (after processing) that fully span the window 12001-13000. Warning: bam file 20430884632_refined_RM.bam has no reads (after processing) that fully span the window 12001-13000. Warning: bam file 20430884732_refined_RM.bam has no reads (after processing) that fully span the window 12001-13000. Warning: bam file 20440569932_refined_RM.bam has no reads (after processing) that fully span the window 12001-13000. WARNING: no bam file had any reads in the window 12001-13000. Skipping to the next window. Now extracting and processing reads in window 13001-14000 Warning: bam file 20430884332_refined_RM.bam has no reads (after processing) that fully span the window 13001-14000. Warning: bam file 20430884632_refined_RM.bam has no reads (after processing) that fully span the window 13001-14000. Warning: bam file 20430884732_refined_RM.bam has no reads (after processing) that fully span the window 13001-14000. Warning: bam file 20440569932_refined_RM.bam has no reads (after processing) that fully span the window 13001-14000. WARNING: no bam file had any reads in the window 13001-14000. Skipping to the next window. Now extracting and processing reads in window 14001-15000 Warning: bam file 20430884332_refined_RM.bam has no reads (after processing) that fully span the window 14001-15000. Warning: bam file 20430884632_refined_RM.bam has no reads (after processing) that fully span the window 14001-15000. Warning: bam file 20430884732_refined_RM.bam has no reads (after processing) that fully span the window 14001-15000. Warning: bam file 20440569932_refined_RM.bam has no reads (after processing) that fully span the window 14001-15000. WARNING: no bam file had any reads in the window 14001-15000. Skipping to the next window. Now extracting and processing reads in window 15001-16000 Warning: bam file 20430884332_refined_RM.bam has no reads (after processing) that fully span the window 15001-16000. Warning: bam file 20430884632_refined_RM.bam has no reads (after processing) that fully span the window 15001-16000. Warning: bam file 20430884732_refined_RM.bam has no reads (after processing) that fully span the window 15001-16000. Warning: bam file 20440569932_refined_RM.bam has no reads (after processing) that fully span the window 15001-16000. WARNING: no bam file had any reads in the window 15001-16000. Skipping to the next window. Now extracting and processing reads in window 16001-17000 Warning: bam file 20430884332_refined_RM.bam has no reads (after processing) that fully span the window 16001-17000. Warning: bam file 20430884632_refined_RM.bam has no reads (after processing) that fully span the window 16001-17000. Warning: bam file 20430884732_refined_RM.bam has no reads (after processing) that fully span the window 16001-17000. Warning: bam file 20440569932_refined_RM.bam has no reads (after processing) that fully span the window 16001-17000. WARNING: no bam file had any reads in the window 16001-17000. Skipping to the next window. Now extracting and processing reads in window 17001-18000 Warning: bam file 20430884332_refined_RM.bam has no reads (after processing) that fully span the window 17001-18000. Warning: bam file 20430884632_refined_RM.bam has no reads (after processing) that fully span the window 17001-18000. Warning: bam file 20430884732_refined_RM.bam has no reads (after processing) that fully span the window 17001-18000. Warning: bam file 20440569932_refined_RM.bam has no reads (after processing) that fully span the window 17001-18000. WARNING: no bam file had any reads in the window 17001-18000. Skipping to the next window. Now extracting and processing reads in window 18001-19000 Warning: bam file 20430884332_refined_RM.bam has no reads (after processing) that fully span the window 18001-19000. Warning: bam file 20430884632_refined_RM.bam has no reads (after processing) that fully span the window 18001-19000. Warning: bam file 20430884732_refined_RM.bam has no reads (after processing) that fully span the window 18001-19000. Warning: bam file 20440569932_refined_RM.bam has no reads (after processing) that fully span the window 18001-19000. WARNING: no bam file had any reads in the window 18001-19000. Skipping to the next window. Now extracting and processing reads in window 19001-20000 Warning: bam file 20430884332_refined_RM.bam has no reads (after processing) that fully span the window 19001-20000. Warning: bam file 20430884632_refined_RM.bam has no reads (after processing) that fully span the window 19001-20000. Warning: bam file 20430884732_refined_RM.bam has no reads (after processing) that fully span the window 19001-20000. Warning: bam file 20440569932_refined_RM.bam has no reads (after processing) that fully span the window 19001-20000. WARNING: no bam file had any reads in the window 19001-20000. Skipping to the next window. Now extracting and processing reads in window 20001-21000 Warning: bam file 20430884332_refined_RM.bam has no reads (after processing) that fully span the window 20001-21000. Warning: bam file 20430884632_refined_RM.bam has no reads (after processing) that fully span the window 20001-21000. Warning: bam file 20430884732_refined_RM.bam has no reads (after processing) that fully span the window 20001-21000. Warning: bam file 20440569932_refined_RM.bam has no reads (after processing) that fully span the window 20001-21000. WARNING: no bam file had any reads in the window 20001-21000. Skipping to the next window. Now extracting and processing reads in window 21001-22000 Warning: bam file 20430884332_refined_RM.bam has no reads (after processing) that fully span the window 21001-22000. Warning: bam file 20430884632_refined_RM.bam has no reads (after processing) that fully span the window 21001-22000. Warning: bam file 20430884732_refined_RM.bam has no reads (after processing) that fully span the window 21001-22000. Warning: bam file 20440569932_refined_RM.bam has no reads (after processing) that fully span the window 21001-22000. WARNING: no bam file had any reads in the window 21001-22000. Skipping to the next window. Now extracting and processing reads in window 22001-23000 Warning: bam file 20430884332_refined_RM.bam has no reads (after processing) that fully span the window 22001-23000. Warning: bam file 20430884632_refined_RM.bam has no reads (after processing) that fully span the window 22001-23000. Warning: bam file 20430884732_refined_RM.bam has no reads (after processing) that fully span the window 22001-23000. Warning: bam file 20440569932_refined_RM.bam has no reads (after processing) that fully span the window 22001-23000. WARNING: no bam file had any reads in the window 22001-23000. Skipping to the next window. Now extracting and processing reads in window 23001-24000 Warning: bam file 20430884332_refined_RM.bam has no reads (after processing) that fully span the window 23001-24000. Warning: bam file 20430884632_refined_RM.bam has no reads (after processing) that fully span the window 23001-24000. Warning: bam file 20430884732_refined_RM.bam has no reads (after processing) that fully span the window 23001-24000. Warning: bam file 20440569932_refined_RM.bam has no reads (after processing) that fully span the window 23001-24000. WARNING: no bam file had any reads in the window 23001-24000. Skipping to the next window. Now extracting and processing reads in window 24001-25000 Warning: bam file 20430884332_refined_RM.bam has no reads (after processing) that fully span the window 24001-25000. Warning: bam file 20430884632_refined_RM.bam has no reads (after processing) that fully span the window 24001-25000. Warning: bam file 20430884732_refined_RM.bam has no reads (after processing) that fully span the window 24001-25000. Warning: bam file 20440569932_refined_RM.bam has no reads (after processing) that fully span the window 24001-25000. WARNING: no bam file had any reads in the window 24001-25000. Skipping to the next window. Now extracting and processing reads in window 25001-26000 Warning: bam file 20430884332_refined_RM.bam has no reads (after processing) that fully span the window 25001-26000. Warning: bam file 20430884632_refined_RM.bam has no reads (after processing) that fully span the window 25001-26000. Warning: bam file 20430884732_refined_RM.bam has no reads (after processing) that fully span the window 25001-26000. Warning: bam file 20440569932_refined_RM.bam has no reads (after processing) that fully span the window 25001-26000. WARNING: no bam file had any reads in the window 25001-26000. Skipping to the next window. Now extracting and processing reads in window 26001-27000 Warning: bam file 20430884332_refined_RM.bam has no reads (after processing) that fully span the window 26001-27000. Warning: bam file 20430884632_refined_RM.bam has no reads (after processing) that fully span the window 26001-27000. Warning: bam file 20430884732_refined_RM.bam has no reads (after processing) that fully span the window 26001-27000. Warning: bam file 20440569932_refined_RM.bam has no reads (after processing) that fully span the window 26001-27000. WARNING: no bam file had any reads in the window 26001-27000. Skipping to the next window. Now extracting and processing reads in window 27001-28000 Warning: bam file 20430884332_refined_RM.bam has no reads (after processing) that fully span the window 27001-28000. Warning: bam file 20430884632_refined_RM.bam has no reads (after processing) that fully span the window 27001-28000. Warning: bam file 20430884732_refined_RM.bam has no reads (after processing) that fully span the window 27001-28000. Warning: bam file 20440569932_refined_RM.bam has no reads (after processing) that fully span the window 27001-28000. WARNING: no bam file had any reads in the window 27001-28000. Skipping to the next window. Now extracting and processing reads in window 28001-29000 Warning: bam file 20430884332_refined_RM.bam has no reads (after processing) that fully span the window 28001-29000. Warning: bam file 20430884632_refined_RM.bam has no reads (after processing) that fully span the window 28001-29000. Warning: bam file 20430884732_refined_RM.bam has no reads (after processing) that fully span the window 28001-29000. Warning: bam file 20440569932_refined_RM.bam has no reads (after processing) that fully span the window 28001-29000. WARNING: no bam file had any reads in the window 28001-29000. Skipping to the next window. Info: phyloscanner_make_trees.py has processed all windows but has not produced any trees, either because you told it not to, or because of a lack of reads, or because of non-fatal errors. Check earlier warning/error messages.

Does anyone have any idea of why my bam files have no reads?

PS: I checked the files (e.g. IGV and they are readable)

Thank you for your support! Daniele

ChrisHIV commented 1 year ago

Hi Daniele, here are some reasons why phyloscanner windows might contain no reads even though your bam contains reads:

Daniele-Pantano commented 1 year ago

Thank you for your reply. I had a chat with the colleague who took care of the sequencing and he told me that as the earlier Easyseq assays were ran at 300bp (so twice 150bp) I should try a window of 150. Now, it is working: I got my RAxML trees generated and I will proceed with the rest of the analysis.

ChrisHIV commented 1 year ago

For paired-read data, phyloscanner can merge the two reads in pair when they overlap with each other i.e. if the size of inserts/fragments is sometimes less than twice the read length. How good your phyloscanner results are depends strongly on the window width, so reading section 3.2 of the manual and using the tools it explains to investigate your data is recommended. Good luck