phyluce_snp_phase_uces - empty joined_allele_sequences_all_samples.fasta

[fspaulding]

I have been trying to run my uce data through phyluce tutorial II, but have encountered an issue. The final output joined_allele_sequences_all_samples.fasta is empty, and only contains headers in the following format:

(base) [fspaulding@vm142-14 fastas]$ head joined_allele_sequences_all_samples.fasta 
>_0 |_phased
>_1 |_phased
>_0 |_phased
>_1 |_phased
>_0 |_phased
>_1 |_phased
>_0 |_phased
>_1 |_phased 

The steps from tutorial II appeared to run fine in terminal, but it seems that my issue is similar to the issues #113 , #117 , and #107 which would have been fixed in the current version (which is what I've installed). I followed a recommendation of trying to run through tutorial II with the data provided from tutorial I in case the issue stems from my own data; however this problem still persists when using the phyluce tutorial data. Looking at the log files, the ones outputted by samtools appear to be empty. My version of samtools:

Program: samtools (Tools for alignments in the SAM format)
Version: 1.10 (using htslib 1.10.2)

Likewise, when looking at the output for samtools-bcftools-out.logcontained the following error:

/home/fspaulding/anaconda2/bin/bcftools: error while loading shared libraries: libcrypto.so.1.0.0: cannot open shared object file: No such file or directory

It seems that the issue may be stemming from samtools, but I'm not sure. Attached are log files from phyluce_snp_bwa_multiple_align, phyluce_snp_phase_uces, and phyluce_align_seqcap_align as well as some of the output files from alligator_mississippiensis. If anyone has an idea of what's going on I would greatly appreciate any feedback on how to potentially resolve this issue.

Thanks,

-Fern

phyluce_align_seqcap_align.log phyluce_snp_bwa_multiple_align.log phyluce_snp_phase_uces.log multialign-bams-logs.zip multialign-bams-phased-reads-logs.zip

[brantfaircloth]

hi!

and, yeah, this is a weird samtools/biopython issue. you basically have to force reinstall samtools.

so something like:

conda remove —force samtools

that’s two dashes before force, then run

conda install samtools

when you are in your phyluce conda environment should do what you need.

-b

On Sep 10, 2020, at 17:13, fspaulding notifications@github.com wrote:

 I have been trying to run my uce data through phyluce tutorial II, but have encountered an issue. The final output joined_allele_sequences_all_samples.fasta is empty, and only contains headers in the following format:

(base) [fspaulding@vm142-14 fastas]$ head joined_allele_sequences_all_samples.fasta

_0 |_phased _1 |_phased _0 |_phased _1 |_phased _0 |_phased _1 |_phased _0 |_phased _1 |_phased The steps from tutorial II appeared to run fine in terminal, but it seems that my issue is similar to the issues #113 , #117 , and #107 which would have been fixed in the current version (which is what I've installed). I followed a recommendation of trying to run through tutorial II with the data provided from tutorial I in case the issue stems from my own data; however this problem still persists when using the phyluce tutorial data. Looking at the log files, the ones outputted by samtools appear to be empty. My version of samtools:

Program: samtools (Tools for alignments in the SAM format) Version: 1.10 (using htslib 1.10.2) Likewise, when looking at the output for samtools-bcftools-out.logcontained the following error:

/home/fspaulding/anaconda2/bin/bcftools: error while loading shared libraries: libcrypto.so.1.0.0: cannot open shared object file: No such file or directory It seems that the issue may be stemming from samtools, but I'm not sure. Attached are log files from phyluce_snp_bwa_multiple_align, phyluce_snp_phase_uces, and phyluce_align_seqcap_align as well as some of the output files from alligator_mississippiensis. If anyone has an idea of what's going on I would greatly appreciate any feedback on how to potentially resolve this issue.

Thanks,

-Fern

phyluce_align_seqcap_align.log phyluce_snp_bwa_multiple_align.log phyluce_snp_phase_uces.log multialign-bams-logs.zip multialign-bams-phased-reads-logs.zip

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[fspaulding]

Hi Brant, thanks for the quick response. I've tried force reinstalling samtools and re-ran the last few steps in the tutorial, but I am still am getting an empty .fasta file. Here's the output from the reinstall:

(base) [fspaulding@vm142-15 all]$ conda remove --force samtools

## Package Plan ##

  environment location: /home/fspaulding/anaconda2

  removed specs:
    - samtools

The following packages will be REMOVED:

  samtools-1.10-h9402c20_2

Proceed ([y]/n)? y

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
(base) [fspaulding@vm142-15 all]$ conda install samtools
Collecting package metadata (current_repodata.json): done
Solving environment: / 
The environment is inconsistent, please check the package plan carefully
The following packages are causing the inconsistency:

  - bioconda/noarch::phyluce==1.6.8=py_0
  - bioconda/linux-64::trinity==2.1.1=6
  - bioconda/linux-64::itero==1.0.1=py27_0
done

## Package Plan ##

  environment location: /home/fspaulding/anaconda2

  added / updated specs:
    - samtools

The following NEW packages will be INSTALLED:

  samtools           bioconda/linux-64::samtools-1.10-h9402c20_2

Proceed ([y]/n)? y

Preparing transaction: done
Verifying transaction: done
Executing transaction: done

Is it possible that the other packages listed that are causing an inconsistency (i.e.trinity, or itero) need to be force reinstalled as well?

-Fern

[brantfaircloth]

Does samtools appear to be working as expected?

[fspaulding]

I believe so, I checked using samtools view and samtools stats (some of the output is below).

(base) [fspaulding@vm142-15 all]$ samtools stats multialign-bams/alligator_mississippiensis/alligator_mississippiensis-CL-RG-MD-M.bam
# This file was produced by samtools stats (1.10+htslib-1.10.2) and can be plotted using plot-bamstats
# This file contains statistics for all reads.
# The command line was:  stats multialign-bams/alligator_mississippiensis/alligator_mississippiensis-CL-RG-MD-M.bam
# CHK, Checksum [2]Read Names   [3]Sequences    [4]Qualities
# CHK, CRC32 of reads which passed filtering followed by addition (32bit overflow)
CHK 601f5bd6    1e645319    648ff73c
# Summary Numbers. Use `grep ^SN | cut -f 2-` to extract this part.
SN  raw total sequences:    3279362
SN  filtered sequences: 0
SN  sequences:  3279362
SN  is sorted:  1
SN  1st fragments:  1705959
SN  last fragments: 1573403
SN  reads mapped:   39731
SN  reads mapped and paired:    23174   # paired-end technology bit set + both mates mapped

[brantfaircloth]

Will you look in the file shown in this commit for your phyluce installation and ensure the test in the right hand side in green is present?

[fspaulding]

I'm new to github and not familiar with it - but I'm looking for the file.

[brantfaircloth]

helpful if I gave you the commit and what to look for: https://github.com/faircloth-lab/phyluce/commit/47201b1a46c077fa509fa5f2ec4683058336b828

so, try which phyluce_snp_phase_uces, then open that file in a text editor (or on command line using less.

[fspaulding]

Okay - here's the contents of phyluce_snp_phase_uceson my machine. It looks like it matches updated one on the commit. Line 34 has #import pdb and line 94 has allele_0_file, allele_1_file = samtools.phase(log, sample, phasing_out_dir, reference, bam).

I'm checking the other files in the commit as well...

[brantfaircloth]

Ok - i'm not sure why it's not working for you. I'll need to some example files to work with, but I also do not have tons of time right now to dedicate to this.

[fspaulding]

No worries - I'll keep chipping away at it and will update this thread if I find out anything else. Thanks for your help!

[wxguillo]

I had the same problem and the following line from one of my samtools-bcftools-out.log files: /home/wxg1/miniconda2/envs/wxg/bin/bcftools: error while loading shared libraries: libcrypto.so.1.0.0: cannot open shared object file: No such file or directory convinced me it was an incompatibility between my versions of samtools and openssl 1.1.1. I used install -c bioconda samtools openssl=1.0 to downgrade openssl to 1.0 and that fixed the problem