Patric and NCBI assmbled isolates

rx32940 commented 5 years ago

Patric database was found from Guglielmini, J., Bourhy, P., Schiettekatte, O., Zinini, F., Brisse, S., & Picardeau, M. (2019). Genus-wide Leptospira core genome multilocus sequence typing for strain taxonomy and global surveillance. PLOS Neglected Tropical Diseases, 13(4), e0007374. https://doi.org/10.1371/journal.pntd.0007374
by searching "Leptospira" in the database, 664 assembled genomes were found.
- out of the 664 genomes, 637 were WGS or complete assembled genomes (~8 were Leptospira phages, rest were plasmids)
- out of 637, 337 has a collection date
in order to build a concatenated genome for each isolate, whole-genome sequences available need to be first assembled into fasta files

rx32940 commented 5 years ago

1251 wgs sra records were extracted from NCBI, 1037 of them are unique isolates
- excluding biosamples overlap with the assemblies from PATRIC and biosamples with collection date, 647 biosamples need to be assembled.
For isolates with collection date: out of 240 sra records were extracted from NCBI, only 182 unique isolates were identified.
By adding unique isolates identified from both sra records and from Patric and also those overlapped in the two databases, I was able to identify 388 unique isolates with the collection date.
- However, out of the 388 isolates, some were assembled records only, some with only raw reads only (51).
  - 1/51 is a not applicable isolate when searched on NCBI: SAMN04721730
    - Therefore only 50 isolates will be assembled with SPADES /scratch/rx32940/assembled_51/{biosample_id}/scaffolds.fasta

rx32940 commented 4 years ago

from the 647 biosamples (sra records) found above that need to be assembled, 644 SRA isolates need to be assembled without a collection date:
- SAMN00009845 was unidentified
- SAMN02580646 has only gene sequences
- SAMN13046976 was found with collection date, so I assembled it and added to the 51 isolates self-assembled with collection date list.

Note: 313 isolates from Picardeu lab in their 2019 paper was also included in the 644 isolates

4 PATRIC assemblies found no assembly records on NCBI, thus 633 PATRIC assemblies (w/ and w/o collection date):
- SAMN01920606
- SAMN01920607
- SAMN01920612
- SAMN01920627
all PATRIC assemblies were downloaded from NCBI's assembly database with their latest version (at 1/22/20) under the folder:
```
/scratch/rx32940/PATRIC_assemblies_633/ncbi-genomes-2020-01-22
```
because all assembly files were named with their assembly accession, I renamed all file to their biosample accession to be consistent (dict for assembly to biosample found from PATRIC metadata table) : script for conversion

rx32940 commented 4 years ago

644 SRA isolates need to be assembled was sequenced with 6 different platform, probably gonna need with three different assemblers
- all picardeau isolates (313 isolates) were store under:
```
/scratch/rx32940/picardeau/SRA_seq/
```

rx32940 commented 4 years ago

Due to sequencing technology 454 discontinued and concerning with the quality and the coverage of the sequencing technology (newbler), we decided to drop all 454 sequenced sra records (only found in no collection date sra records).
SAMN00009845 is a sample taken from metagenome, take out the sequence
most of them were sequenced under an illumina platform
3 isolates were sequenced with Pacbio (use Canu for assembling)

-- we only left with 528 no date isolates (215 excluding the picardeau isolates)

rx32940 commented 4 years ago

PRJEB2095 (104 isolates) from Wellcome Sanger Institute is pre-publication, probably need approval for use https://www.sanger.ac.uk/resources/downloads/bacteria/leptospira-interrogans.html

lsalvador commented 4 years ago

test

rx32940 commented 4 years ago

three Pacbio Runs for non date isolates (SRR6703763, SRR2423280, SRR6703608) from JCVI PRJNA401006 could not be assembled due to bad quality.
Other three isolates from the same project were already assembled (w/o sra records). error message from canu for SRR6703763:
```
Skipped (too short):
24901017 bp (1.1764%).
115285 reads (39.8951%).
gatekeeperCreate did NOT finish successfully; too many short reads.  Check your reads!
gatekeeperCreate finished successfully.
```
currently, 525 isolates without collection date metadata (212 w/o picardeau isolates)

rx32940 commented 4 years ago

12 runs stored in ERX006638 were "Illumina sequencing of barcoded sample library "L interogans 3" for the "Discovery of sequence diversity in Leptospira interrogans ST34" study"

these runs do not account for any isolates, just for sequencing purposes barcoded runs: ERR027302 ERR027303 ERR027304 ERR027305 ERR027306 ERR027307 ERR027308 ERR027309 ERR027310 ERR027311 ERR027312 ERR027313

rx32940 commented 4 years ago

sample SAMN04090012, has 4 technical replicates (SRR2423277, SRR2423278, SRR2423279, SRR2423280). four replicates for the biosample was cat into the same file with the command below
```
cat SRR24232*_1(2).fastq.gz > SAMN04090012_1(2).fastq.gz
```
some of the pair-end fastq sequenced by Wellcome Sanger Institute(SC) were labeled as "_3" or "_4" for the reverse file instead of "_2" at end of the file (before extensions).
the runs with reverse fastq labeled with "_4" also have a file labeled as "_3", however, only reads with single nucleotide was found in these files, thus will not be used for assembly.
runs with "_3" and "_4" labels for reverse file can be find in
```
/scratch/rx32940/rest_sra_216/PairEnd_3(4).txt
```

rx32940 commented 4 years ago

4/212 no date assembled isolates wasn't able to get assmbled:

pipeline didn't finish with the error ERR017078 (SAMEA864154) ERR017080 (SAMEA864155) ERR017165 (SAMEA864093) - coverage size too low

== Error ==  system call for: "['/usr/local/apps/gb/spades/3.12.0-k_245/bin/spades-core', '/scratch/rx32940/rest_sra_216/assemblies/ERR017165/K21/configs/config.info']" finished abnormally, err code: -6

pipeline finished with only contigs, but not scaffolds SAMN04090012

rx32940 commented 4 years ago

all assemblies finished, with 1209 isolates in total (excluding 12 assemblies from barcoded runs found in rest_sra_216 folder, including 4 fail to assemble)

list of biosamples with assemblies available in

/scratch/rx32940/All_Lepto_Assemblies/all_assemblies_1209.txt

all assemblies stored in folders below, under the assemblies sub-dir in the scaffolds.fasta file (the 4 isolates couldn't be assembled has no scaffolds.fasta ), except for PATRIC_assemblies_633, which has assemblies stored in ${biosample_acc}.fna.gz file
```
/scratch/rx32940/All_Lepto_Assemblies/dated_assembled_51(picardeau_313/rest_sra_216/PATRIC_assemblies_633)
```
all assemblies were quasted with the reference genome of Leptospira interrogans serovar Lai found in folder with sequences (.fna) and annotations(.gff) and can be find in the quast_assemblies folder from the dir above
```
/scratch/rx32940/reference
```
species for each isolates were extracted by submitting all_assemblies_1209.txt to Batch Entrez and download full xml from NCBI. 1) organism names were parsed from xml with python script 2) species names were extracted from organism names with R script 3) biosample accessions with organism names and their species can be find in file
```
/Users/rx32940/Dropbox/5.Rachel-projects/Phylogeography/Organism_fullname_1209.txt
```

rx32940 commented 4 years ago

rx32940 / Lepto-Phylogeography

Patric and NCBI assmbled isolates #3

To ensure the quality of the assemblies, start a second version of assembling in the new issue