Project Plan - Githubissues

annamtown commented 6 years ago

My project will analyze Salmonella enterica subsp. enterica serovar Enteritidis (S. enteritidis) genomes from a variety of international locations. I think the paper given below would be a good study to replicate: https://www.nature.com/articles/ng.3644.pdf In the supplementary materials, they provide the accession numbers for 674 isolates of S. enteritidis. One issue regarding these isolates is that they were collected over 63 years. I'm sure sure how much the DNA degraded for older samples. All of the data was sequenced using Illumina HiSeq 2000. I would prefer to work with recent isolates, but this will also decrease the same size.

Depending on how closely I want to replicate this paper, I could repeat their phylogenetic analyses using HierBAPS (and potentially with BEAST), but analyze other aspects of how the clades could be related aside from location of origin and sequence similarity. I've worked on a small CRISPR project before, so maybe I could analyze the CRISPR loci and compare spacers of each isolate within and between clades?

cbergman commented 6 years ago

This looks good, I would focus on a small sample (10-20 strains) to begin with (it doesn't matter which strains) then try to scale up to the whole data set if you can
I wouldn't focus on CRISPR and instead try to replicate the phylogenetic analysis
the trick here will be to leverage the cluster to submit multiple jobs at once, basically this will require writing a script that writes and runs other scripts

cbergman commented 6 years ago

One thing to keep an eye out for is if the data have 1 pair of fastq per sample or >1 pair per sample

annamtown commented 6 years ago

Supplementary Data: https://www.nature.com/articles/ng.3644#supplementary-information

cbergman commented 6 years ago

The project accession in the paper is https://www.ncbi.nlm.nih.gov/bioproject/?term=PRJNA248792, which has over 18,000 runs
There are onbly ~700 samples in the supplemental file
using one of the samples in the supplemental file (ERS255498), I queried EBI and found a differnt project ID (PRJEB634) that has ~700 samples: https://www.ebi.ac.uk/ena/data/view/PRJEB634
you can download a .tsv table of the meta-data for this project here: https://www.ebi.ac.uk/ena/data/warehouse/filereport?accession=PRJEB634&result=read_run&fields=study_accession,sample_accession,secondary_sample_accession,sample_alias,experiment_accession,run_accession,scientific_name,instrument_model,library_layout,library_source,library_selection,read_count,base_count,experiment_title,fastq_ftp
wget this file into your repo and do some analysis with grep to see if all samples are PAIRED and there are only one of each _1.fq.gz and _2.fq.gz files for each sample.

annamtown commented 6 years ago

[x] need to figure out how to search this file 032ef53 to see if samples are paired and that there are two fq.qz files per sample.

Yes, all samples are paired with two fastq reads

[x] review files generated from projectloop.sh to ensure it executed properly

_quast, mummer, and prokka did not run correctly - fixed this issue by removing these commands because they are unneeded for this project

[x] begin script for phylogenetic analyses: RAxML and HierBAPS
will combine script with original script for consensus sequence generation after drafting and troubleshooting

RAxML specifies GTR + I + G model

# run RAxML GTR with + I + G model and 100 bootstrap pseudoreplicate analyses of the alignment data
#-T is number of threads
#-f a is rapid Bootstrap analysis and search for bestscoring ML tree in one program run 
#-G enables the MLbased evolutionary placement algorithm heuristics by specifying a threshold value of 0.1 (10% of branches considered for thorough insertions)
#-I is a posteriori bootstopping analysis
#-x specifies an integer number (random seed) and turn on rapid bootstrapping
#-p specifies a random number seed for the parsimony inferences
#-m is the model
#-n is the name of the new files
#-s is the input file
raxmlHPC-PTHREADS -T 6 -f a -G 0.1 -x 12345 -p 4523 -m GTRGAMMA -I autoFC -n enteritidis -s allconsensus.fasta -# 100

cbergman commented 6 years ago

To convert your multiple consensus.fa files: 1) rename each fasta header:

sed "s/>.*/>$i/g"

2) concatenate into one multi-fasta file

cat /path/to/consensus/*.consenus.fa

3) optional?: convert multi-fasta to phylip using EMBOSS

seqret -sequence fasta::input.fna -outseq phylip::output.phy

annamtown commented 6 years ago

[x] check results to see if consensus files combined into one multi-fasta file Yes, one multi-fasta file was created.

[x] check results to see if RAxML ran properly RAxML did not run properly


RAxML can't, parse the alignment file as phylip file 
it will now try to parse it as FASTA file

Fasta parsing error, RAxML expects an alignment. the sequence before taxon >ERR338265 : seems to have a different length



Will try to convert to .phy and run RAxML again

annamtown commented 6 years ago

RAxML ran successfully after converting my multi-fasta file to phylip format:

# convert allconsensus.fasta to .phy format
/usr/local/emboss/latest/bin/seqret -sequence fasta::allconsensus.fasta -outseq phylip::allconsensus.phy

Successfully ran RAxML with phylip file:

/usr/local/raxml/latest/raxmlHPC-PTHREADS -T 6 -f a -G 0.1 -x 12345 -p 4523 -m GTRGAMMA -I autoFC --no-bfgs -n enteritidis -s allconsensus.phy -# 100

However, I am uncertain if these exact options were used in the analyses completed in the paper. The paper states, "A maximum-likelihood phylogenetic tree was then built from the alignments of the isolates using RAxML (version 7.0.4) with a GTR + I + G model." I read the manual and guessed which parameter to use for the -I and -G flags.

cbergman commented 6 years ago

Good news, time to scale up to more samples and see if everything works. If it does, you can think about trying to progam a more complex version of the pipeline that submits multiple jobs to the cluster
You can visualize the tree in FigTree: http://tree.bio.ed.ac.uk/software/figtree/
Can you post a version of the tree here?
The relevant file should be something like RAxML_bipartitions.enteritidis

The GTR + I + G model is specified using the -m flag only

/usr/local/raxml/latest/raxmlHPC-PTHREADS -T 6 -f a -x 12345 -p 4523 -m GTRGAMMAI -n enteritidis -s allconsensus.phy -# 100

annamtown commented 6 years ago

changed GTR + I + G model command to

/usr/local/raxml/latest/raxmlHPC-PTHREADS -T 6 -f a -x 12345 -p 4523 -m GTRGAMMAI -n enteritidis -s allconsensus.phy -# 100

figured out a way to include the wget command in the loop:

wget -q -O ${i}_1.fastq.gz ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR369/${i:0:6}/${i}_1.fastq.gz
wget -q -O ${i}_2.fastq.gz ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR369/${i:0:6}/${i}_2.fastq.gz

Tree files from 10 sample run

bipartitions tree file: https://github.com/annamtown/gene8940/blob/master/RAxML_bipartitions.enteritidis
bestTree file: https://github.com/annamtown/gene8940/blob/master/RAxML_bestTree.enteritidis

I can also post images/PDFs of the trees from FigTree here, if you want me to.

I just submitted a new job for 50 isolates to see if my new wget command works for downloading the isolates from ENA. I will post trees from this job when it is finished. Job did not run correctly. I got a bunch of "core.____" files in my directory. What does this mean? This is the script I used: 4f70f93 I assume this is a core dump, so I increased my thread count from four to six.

cbergman commented 6 years ago

I think you need to change:

wget -q -O ${i}_1.fastq.gz ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR369/${i:0:6}/${i}_1.fastq.gz
wget -q -O ${i}_2.fastq.gz ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR369/${i:0:6}/${i}_2.fastq.gz

to

wget -q -O ${i}_1.fastq.gz ftp://ftp.sra.ebi.ac.uk/vol1/fastq/${i:0:6}/${i}/${i}_1.fastq.gz
wget -q -O ${i}_2.fastq.gz ftp://ftp.sra.ebi.ac.uk/vol1/fastq/${i:0:6}/${i}/${i}_2.fastq.gz

I would try to work on the original 10 samples till you get the auto-download fixed
yes please post images of trees when you get them

annamtown commented 6 years ago

I ran 10 samples with the wget command in the loop. It worked correctly.
Last Tuesday, I submitted a 50 sample job. It is still running. Should I cancel this job? I'm only missing three RAxML files. It appears that every other command has worked properly. I used:
```
grep -c "ERR" allconsensus.phy
```
and the output was 50, so I know that all of my files uploaded successfully with the new wget command.

cbergman commented 6 years ago

this looks like raxml is hanging
do you know how long it took to run raxml on 10 samples?
one problem might be that you asked SGE for 4 cores, but you are using 6 cores for the raxml run
another problem might be that you have old output files for ramxl in the output directory that are causing problems
do you see any intermediate output files for the ramxl run that suggest it is still working?
could you ls -lrt the directory with the raxml results and post the results here

annamtown commented 6 years ago

My 10 sample job ran for approximately 24 hours, possibly fewer

I removed all the of files from the "project2" directory prior to running the job

total 63358206
drwx------ 12 s_11 student        12 Nov 20 09:10 ..
-rw-r--r--  1 s_11 student   1455509 Nov 21 12:09 ref.fa.gz
-rw-r--r--  1 s_11 student    255004 Nov 21 12:09 meta.tsv
-rw-r--r--  1 s_11 student   4764020 Nov 21 12:09 ref.fa
-rw-r--r--  1 s_11 student   4685940 Nov 21 12:09 ref.fa.bwt
-rw-r--r--  1 s_11 student   1171464 Nov 21 12:09 ref.fa.pac
-rw-r--r--  1 s_11 student       100 Nov 21 12:09 ref.fa.ann
-rw-r--r--  1 s_11 student        12 Nov 21 12:09 ref.fa.amb
-rw-r--r--  1 s_11 student   2342976 Nov 21 12:09 ref.fa.sa
-rw-r--r--  1 s_11 student 143790020 Nov 21 12:10 ERR369378_1.fastq.gz
-rw-r--r--  1 s_11 student 152036560 Nov 21 12:11 ERR369378_2.fastq.gz
-rw-r--r--  1 s_11 student 311751920 Nov 21 12:13 ERR369378.aln.bam
-rw-r--r--  1 s_11 student 298928143 Nov 21 12:14 ERR369378.aln.sort.bam
-rw-r--r--  1 s_11 student     14200 Nov 21 12:14 ERR369378.aln.sort.bam.bai
-rw-r--r--  1 s_11 student        28 Nov 21 12:14 ref.fa.fai
-rw-r--r--  1 s_11 student  44066815 Nov 21 12:23 ERR369378.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3465 Nov 21 12:23 ERR369378.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763995 Nov 21 12:23 ERR369378.consensus.fa
-rw-r--r--  1 s_11 student 149601225 Nov 21 12:24 ERR369348_1.fastq.gz
-rw-r--r--  1 s_11 student 158585695 Nov 21 12:25 ERR369348_2.fastq.gz
-rw-r--r--  1 s_11 student 324167810 Nov 21 12:28 ERR369348.aln.bam
-rw-r--r--  1 s_11 student 309745935 Nov 21 12:28 ERR369348.aln.sort.bam
-rw-r--r--  1 s_11 student     14280 Nov 21 12:29 ERR369348.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  45973583 Nov 21 12:36 ERR369348.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3430 Nov 21 12:37 ERR369348.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763964 Nov 21 12:37 ERR369348.consensus.fa
-rw-r--r--  1 s_11 student 166800216 Nov 21 12:38 ERR338264_1.fastq.gz
-rw-r--r--  1 s_11 student 169726518 Nov 21 12:39 ERR338264_2.fastq.gz
-rw-r--r--  1 s_11 student 396938163 Nov 21 12:41 ERR338264.aln.bam
-rw-r--r--  1 s_11 student 381090094 Nov 21 12:42 ERR338264.aln.sort.bam
-rw-r--r--  1 s_11 student     14320 Nov 21 12:42 ERR338264.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  46184225 Nov 21 12:52 ERR338264.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3441 Nov 21 12:52 ERR338264.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763953 Nov 21 12:53 ERR338264.consensus.fa
-rw-r--r--  1 s_11 student 172155330 Nov 21 12:54 ERR338265_1.fastq.gz
-rw-r--r--  1 s_11 student 175009685 Nov 21 12:55 ERR338265_2.fastq.gz
-rw-r--r--  1 s_11 student 406754235 Nov 21 12:58 ERR338265.aln.bam
-rw-r--r--  1 s_11 student 389641025 Nov 21 12:58 ERR338265.aln.sort.bam
-rw-r--r--  1 s_11 student     14408 Nov 21 12:59 ERR338265.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  42944445 Nov 21 13:09 ERR338265.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3460 Nov 21 13:09 ERR338265.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763963 Nov 21 13:09 ERR338265.consensus.fa
-rw-r--r--  1 s_11 student 175707602 Nov 21 13:11 ERR338272_1.fastq.gz
-rw-r--r--  1 s_11 student 178307474 Nov 21 13:12 ERR338272_2.fastq.gz
-rw-r--r--  1 s_11 student 414719859 Nov 21 13:14 ERR338272.aln.bam
-rw-r--r--  1 s_11 student 397208336 Nov 21 13:15 ERR338272.aln.sort.bam
-rw-r--r--  1 s_11 student     14288 Nov 21 13:15 ERR338272.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  42853820 Nov 21 13:26 ERR338272.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3434 Nov 21 13:26 ERR338272.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763958 Nov 21 13:26 ERR338272.consensus.fa
-rw-r--r--  1 s_11 student 180266270 Nov 21 13:27 ERR338273_1.fastq.gz
-rw-r--r--  1 s_11 student 183186352 Nov 21 13:28 ERR338273_2.fastq.gz
-rw-r--r--  1 s_11 student 426246268 Nov 21 13:31 ERR338273.aln.bam
-rw-r--r--  1 s_11 student 408559472 Nov 21 13:32 ERR338273.aln.sort.bam
-rw-r--r--  1 s_11 student     14328 Nov 21 13:32 ERR338273.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  42811214 Nov 21 13:43 ERR338273.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3420 Nov 21 13:43 ERR338273.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763958 Nov 21 13:43 ERR338273.consensus.fa
-rw-r--r--  1 s_11 student 160775594 Nov 21 13:44 ERR338280_1.fastq.gz
-rw-r--r--  1 s_11 student 163665796 Nov 21 13:45 ERR338280_2.fastq.gz
-rw-r--r--  1 s_11 student 380102816 Nov 21 13:48 ERR338280.aln.bam
-rw-r--r--  1 s_11 student 364097216 Nov 21 13:48 ERR338280.aln.sort.bam
-rw-r--r--  1 s_11 student     14328 Nov 21 13:48 ERR338280.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  43900980 Nov 21 13:58 ERR338280.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3447 Nov 21 13:58 ERR338280.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763968 Nov 21 13:59 ERR338280.consensus.fa
-rw-r--r--  1 s_11 student 149055872 Nov 21 14:00 ERR338281_1.fastq.gz
-rw-r--r--  1 s_11 student 151835310 Nov 21 14:01 ERR338281_2.fastq.gz
-rw-r--r--  1 s_11 student 352612204 Nov 21 14:03 ERR338281.aln.bam
-rw-r--r--  1 s_11 student 338106591 Nov 21 14:03 ERR338281.aln.sort.bam
-rw-r--r--  1 s_11 student     14256 Nov 21 14:03 ERR338281.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  44608338 Nov 21 14:13 ERR338281.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3422 Nov 21 14:13 ERR338281.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763963 Nov 21 14:13 ERR338281.consensus.fa
-rw-r--r--  1 s_11 student 150096732 Nov 21 14:14 ERR338288_1.fastq.gz
-rw-r--r--  1 s_11 student 152911541 Nov 21 14:15 ERR338288_2.fastq.gz
-rw-r--r--  1 s_11 student 353467838 Nov 21 14:17 ERR338288.aln.bam
-rw-r--r--  1 s_11 student 339179148 Nov 21 14:17 ERR338288.aln.sort.bam
-rw-r--r--  1 s_11 student     14264 Nov 21 14:17 ERR338288.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  45264457 Nov 21 14:26 ERR338288.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3435 Nov 21 14:26 ERR338288.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763958 Nov 21 14:27 ERR338288.consensus.fa
-rw-r--r--  1 s_11 student 166438160 Nov 21 14:28 ERR338255_1.fastq.gz
-rw-r--r--  1 s_11 student 169153906 Nov 21 14:29 ERR338255_2.fastq.gz
-rw-r--r--  1 s_11 student 391997864 Nov 21 14:31 ERR338255.aln.bam
-rw-r--r--  1 s_11 student 376626341 Nov 21 14:32 ERR338255.aln.sort.bam
-rw-r--r--  1 s_11 student     14168 Nov 21 14:32 ERR338255.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  43072331 Nov 21 14:42 ERR338255.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3470 Nov 21 14:42 ERR338255.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4764000 Nov 21 14:42 ERR338255.consensus.fa
-rw-r--r--  1 s_11 student 145415243 Nov 21 14:43 ERR338296_1.fastq.gz
-rw-r--r--  1 s_11 student 148113510 Nov 21 14:43 ERR338296_2.fastq.gz
-rw-r--r--  1 s_11 student 343251709 Nov 21 14:45 ERR338296.aln.bam
-rw-r--r--  1 s_11 student 329488490 Nov 21 14:46 ERR338296.aln.sort.bam
-rw-r--r--  1 s_11 student     14200 Nov 21 14:46 ERR338296.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  45174559 Nov 21 14:55 ERR338296.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3426 Nov 21 14:55 ERR338296.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763970 Nov 21 14:55 ERR338296.consensus.fa
-rw-r--r--  1 s_11 student 169321568 Nov 21 14:57 ERR338256_1.fastq.gz
-rw-r--r--  1 s_11 student 171979659 Nov 21 14:58 ERR338256_2.fastq.gz
-rw-r--r--  1 s_11 student 400232468 Nov 21 15:00 ERR338256.aln.bam
-rw-r--r--  1 s_11 student 383529970 Nov 21 15:01 ERR338256.aln.sort.bam
-rw-r--r--  1 s_11 student     14320 Nov 21 15:01 ERR338256.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  42956471 Nov 21 15:11 ERR338256.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3433 Nov 21 15:12 ERR338256.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763973 Nov 21 15:12 ERR338256.consensus.fa
-rw-r--r--  1 s_11 student 149993103 Nov 21 15:13 ERR338304_1.fastq.gz
-rw-r--r--  1 s_11 student 152750151 Nov 21 15:14 ERR338304_2.fastq.gz
-rw-r--r--  1 s_11 student 354184393 Nov 21 15:16 ERR338304.aln.bam
-rw-r--r--  1 s_11 student 339902672 Nov 21 15:17 ERR338304.aln.sort.bam
-rw-r--r--  1 s_11 student     14280 Nov 21 15:17 ERR338304.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  44691667 Nov 21 15:26 ERR338304.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3451 Nov 21 15:26 ERR338304.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763958 Nov 21 15:26 ERR338304.consensus.fa
-rw-r--r--  1 s_11 student 191527793 Nov 21 15:27 ERR338257_1.fastq.gz
-rw-r--r--  1 s_11 student 194338997 Nov 21 15:29 ERR338257_2.fastq.gz
-rw-r--r--  1 s_11 student 452965695 Nov 21 15:31 ERR338257.aln.bam
-rw-r--r--  1 s_11 student 434157732 Nov 21 15:32 ERR338257.aln.sort.bam
-rw-r--r--  1 s_11 student     14416 Nov 21 15:32 ERR338257.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  42542631 Nov 21 15:44 ERR338257.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3423 Nov 21 15:44 ERR338257.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763959 Nov 21 15:44 ERR338257.consensus.fa
-rw-r--r--  1 s_11 student 156332762 Nov 21 15:45 ERR338914_1.fastq.gz
-rw-r--r--  1 s_11 student 162921449 Nov 21 15:46 ERR338914_2.fastq.gz
-rw-r--r--  1 s_11 student 372992204 Nov 21 15:48 ERR338914.aln.bam
-rw-r--r--  1 s_11 student 359253266 Nov 21 15:49 ERR338914.aln.sort.bam
-rw-r--r--  1 s_11 student     14184 Nov 21 15:49 ERR338914.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  45985412 Nov 21 15:58 ERR338914.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3440 Nov 21 15:58 ERR338914.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4764011 Nov 21 15:58 ERR338914.consensus.fa
-rw-r--r--  1 s_11 student 163732550 Nov 21 15:59 ERR338922_1.fastq.gz
-rw-r--r--  1 s_11 student 170674433 Nov 21 16:01 ERR338922_2.fastq.gz
-rw-r--r--  1 s_11 student 391609203 Nov 21 16:03 ERR338922.aln.bam
-rw-r--r--  1 s_11 student 376886168 Nov 21 16:03 ERR338922.aln.sort.bam
-rw-r--r--  1 s_11 student     14320 Nov 21 16:04 ERR338922.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  45273829 Nov 21 16:13 ERR338922.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3453 Nov 21 16:13 ERR338922.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763985 Nov 21 16:14 ERR338922.consensus.fa
-rw-r--r--  1 s_11 student 362675421 Nov 21 16:16 ERR338929_1.fastq.gz
-rw-r--r--  1 s_11 student 377002396 Nov 21 16:18 ERR338929_2.fastq.gz
-rw-r--r--  1 s_11 student 869175223 Nov 21 16:23 ERR338929.aln.bam
-rw-r--r--  1 s_11 student 832072912 Nov 21 16:25 ERR338929.aln.sort.bam
-rw-r--r--  1 s_11 student     14992 Nov 21 16:25 ERR338929.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  52392283 Nov 21 16:48 ERR338929.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3440 Nov 21 16:48 ERR338929.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763959 Nov 21 16:48 ERR338929.consensus.fa
-rw-r--r--  1 s_11 student 179963265 Nov 21 16:49 ERR338930_1.fastq.gz
-rw-r--r--  1 s_11 student 187318249 Nov 21 16:50 ERR338930_2.fastq.gz
-rw-r--r--  1 s_11 student 430757261 Nov 21 16:53 ERR338930.aln.bam
-rw-r--r--  1 s_11 student 413998824 Nov 21 16:53 ERR338930.aln.sort.bam
-rw-r--r--  1 s_11 student     14392 Nov 21 16:54 ERR338930.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  44584710 Nov 21 17:04 ERR338930.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3438 Nov 21 17:04 ERR338930.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763958 Nov 21 17:05 ERR338930.consensus.fa
-rw-r--r--  1 s_11 student 162472327 Nov 21 17:06 ERR338956_1.fastq.gz
-rw-r--r--  1 s_11 student 168828518 Nov 21 17:07 ERR338956_2.fastq.gz
-rw-r--r--  1 s_11 student 371958267 Nov 21 17:09 ERR338956.aln.bam
-rw-r--r--  1 s_11 student 359924722 Nov 21 17:09 ERR338956.aln.sort.bam
-rw-r--r--  1 s_11 student     14280 Nov 21 17:09 ERR338956.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  47764179 Nov 21 17:16 ERR338956.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3439 Nov 21 17:16 ERR338956.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763958 Nov 21 17:17 ERR338956.consensus.fa
-rw-r--r--  1 s_11 student 189085014 Nov 21 17:18 ERR338957_1.fastq.gz
-rw-r--r--  1 s_11 student 197237526 Nov 21 17:19 ERR338957_2.fastq.gz
-rw-r--r--  1 s_11 student 452051362 Nov 21 17:22 ERR338957.aln.bam
-rw-r--r--  1 s_11 student 434524338 Nov 21 17:22 ERR338957.aln.sort.bam
-rw-r--r--  1 s_11 student     14352 Nov 21 17:22 ERR338957.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  44599868 Nov 21 17:34 ERR338957.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3467 Nov 21 17:34 ERR338957.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763959 Nov 21 17:34 ERR338957.consensus.fa
-rw-r--r--  1 s_11 student 208828522 Nov 21 17:35 ERR338959_1.fastq.gz
-rw-r--r--  1 s_11 student 217343215 Nov 21 17:36 ERR338959_2.fastq.gz
-rw-r--r--  1 s_11 student 500717516 Nov 21 17:39 ERR338959.aln.bam
-rw-r--r--  1 s_11 student 479933726 Nov 21 17:40 ERR338959.aln.sort.bam
-rw-r--r--  1 s_11 student     14384 Nov 21 17:40 ERR338959.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  44621040 Nov 21 17:52 ERR338959.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3407 Nov 21 17:53 ERR338959.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763966 Nov 21 17:53 ERR338959.consensus.fa
-rw-r--r--  1 s_11 student 165234888 Nov 21 17:54 ERR338960_1.fastq.gz
-rw-r--r--  1 s_11 student 172000500 Nov 21 17:55 ERR338960_2.fastq.gz
-rw-r--r--  1 s_11 student 394949988 Nov 21 17:57 ERR338960.aln.bam
-rw-r--r--  1 s_11 student 380229462 Nov 21 17:57 ERR338960.aln.sort.bam
-rw-r--r--  1 s_11 student     14424 Nov 21 17:58 ERR338960.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  44977182 Nov 21 18:08 ERR338960.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3443 Nov 21 18:08 ERR338960.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763969 Nov 21 18:08 ERR338960.consensus.fa
-rw-r--r--  1 s_11 student 186819782 Nov 21 18:09 ERR338967_1.fastq.gz
-rw-r--r--  1 s_11 student 194418721 Nov 21 18:11 ERR338967_2.fastq.gz
-rw-r--r--  1 s_11 student 447098474 Nov 21 18:13 ERR338967.aln.bam
-rw-r--r--  1 s_11 student 429489253 Nov 21 18:14 ERR338967.aln.sort.bam
-rw-r--r--  1 s_11 student     14496 Nov 21 18:14 ERR338967.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  43907682 Nov 21 18:25 ERR338967.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3592 Nov 21 18:25 ERR338967.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763960 Nov 21 18:26 ERR338967.consensus.fa
-rw-r--r--  1 s_11 student 175369470 Nov 21 18:27 ERR338968_1.fastq.gz
-rw-r--r--  1 s_11 student 182696434 Nov 21 18:28 ERR338968_2.fastq.gz
-rw-r--r--  1 s_11 student 419476104 Nov 21 18:30 ERR338968.aln.bam
-rw-r--r--  1 s_11 student 403306260 Nov 21 18:31 ERR338968.aln.sort.bam
-rw-r--r--  1 s_11 student     14320 Nov 21 18:31 ERR338968.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  44407583 Nov 21 18:41 ERR338968.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3438 Nov 21 18:41 ERR338968.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763960 Nov 21 18:42 ERR338968.consensus.fa
-rw-r--r--  1 s_11 student 213778686 Nov 21 18:43 ERR338969_1.fastq.gz
-rw-r--r--  1 s_11 student 222643717 Nov 21 18:44 ERR338969_2.fastq.gz
-rw-r--r--  1 s_11 student 511941446 Nov 21 18:47 ERR338969.aln.bam
-rw-r--r--  1 s_11 student 490376375 Nov 21 18:48 ERR338969.aln.sort.bam
-rw-r--r--  1 s_11 student     14472 Nov 21 18:48 ERR338969.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  44865244 Nov 21 19:01 ERR338969.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3444 Nov 21 19:01 ERR338969.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763966 Nov 21 19:01 ERR338969.consensus.fa
-rw-r--r--  1 s_11 student 190399431 Nov 21 19:02 ERR338970_1.fastq.gz
-rw-r--r--  1 s_11 student 198465591 Nov 21 19:03 ERR338970_2.fastq.gz
-rw-r--r--  1 s_11 student 456937598 Nov 21 19:06 ERR338970.aln.bam
-rw-r--r--  1 s_11 student 438663704 Nov 21 19:06 ERR338970.aln.sort.bam
-rw-r--r--  1 s_11 student     14472 Nov 21 19:07 ERR338970.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  44725112 Nov 21 19:18 ERR338970.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3449 Nov 21 19:18 ERR338970.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763958 Nov 21 19:18 ERR338970.consensus.fa
-rw-r--r--  1 s_11 student 204952420 Nov 21 19:20 ERR338913_1.fastq.gz
-rw-r--r--  1 s_11 student 213125562 Nov 21 19:21 ERR338913_2.fastq.gz
-rw-r--r--  1 s_11 student 490649474 Nov 21 19:24 ERR338913.aln.bam
-rw-r--r--  1 s_11 student 470643659 Nov 21 19:24 ERR338913.aln.sort.bam
-rw-r--r--  1 s_11 student     14520 Nov 21 19:24 ERR338913.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  44540723 Nov 21 19:37 ERR338913.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3427 Nov 21 19:37 ERR338913.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763962 Nov 21 19:37 ERR338913.consensus.fa
-rw-r--r--  1 s_11 student 101190113 Nov 21 19:38 ERR353345_1.fastq.gz
-rw-r--r--  1 s_11 student 108239539 Nov 21 19:38 ERR353345_2.fastq.gz
-rw-r--r--  1 s_11 student 224337661 Nov 21 19:40 ERR353345.aln.bam
-rw-r--r--  1 s_11 student 214779634 Nov 21 19:40 ERR353345.aln.sort.bam
-rw-r--r--  1 s_11 student     14136 Nov 21 19:40 ERR353345.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  44155591 Nov 21 19:47 ERR353345.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3412 Nov 21 19:47 ERR353345.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763969 Nov 21 19:48 ERR353345.consensus.fa
-rw-r--r--  1 s_11 student 145198816 Nov 21 19:48 ERR374229_1.fastq.gz
-rw-r--r--  1 s_11 student 151031259 Nov 21 19:49 ERR374229_2.fastq.gz
-rw-r--r--  1 s_11 student 312079703 Nov 21 19:51 ERR374229.aln.bam
-rw-r--r--  1 s_11 student 300987204 Nov 21 19:52 ERR374229.aln.sort.bam
-rw-r--r--  1 s_11 student     14280 Nov 21 19:52 ERR374229.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  47318138 Nov 21 20:00 ERR374229.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3423 Nov 21 20:00 ERR374229.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763946 Nov 21 20:00 ERR374229.consensus.fa
-rw-r--r--  1 s_11 student 246976065 Nov 21 20:01 ERR311263_1.fastq.gz
-rw-r--r--  1 s_11 student 267780327 Nov 21 20:02 ERR311263_2.fastq.gz
-rw-r--r--  1 s_11 student 608242327 Nov 21 20:06 ERR311263.aln.bam
-rw-r--r--  1 s_11 student 583037481 Nov 21 20:07 ERR311263.aln.sort.bam
-rw-r--r--  1 s_11 student     14840 Nov 21 20:07 ERR311263.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  47165063 Nov 21 20:23 ERR311263.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3418 Nov 21 20:23 ERR311263.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763927 Nov 21 20:23 ERR311263.consensus.fa
-rw-r--r--  1 s_11 student 264108374 Nov 21 20:24 ERR311256_1.fastq.gz
-rw-r--r--  1 s_11 student 287091640 Nov 21 20:25 ERR311256_2.fastq.gz
-rw-r--r--  1 s_11 student 651932641 Nov 21 20:29 ERR311256.aln.bam
-rw-r--r--  1 s_11 student 624466417 Nov 21 20:30 ERR311256.aln.sort.bam
-rw-r--r--  1 s_11 student     14776 Nov 21 20:30 ERR311256.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  48880588 Nov 21 20:47 ERR311256.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3454 Nov 21 20:47 ERR311256.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763941 Nov 21 20:47 ERR311256.consensus.fa
-rw-r--r--  1 s_11 student 303910235 Nov 21 20:49 ERR311257_1.fastq.gz
-rw-r--r--  1 s_11 student 329361681 Nov 21 20:51 ERR311257_2.fastq.gz
-rw-r--r--  1 s_11 student 747324238 Nov 21 20:55 ERR311257.aln.bam
-rw-r--r--  1 s_11 student 717532888 Nov 21 20:56 ERR311257.aln.sort.bam
-rw-r--r--  1 s_11 student     15080 Nov 21 20:57 ERR311257.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  48697279 Nov 21 21:16 ERR311257.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3456 Nov 21 21:16 ERR311257.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763953 Nov 21 21:16 ERR311257.consensus.fa
-rw-r--r--  1 s_11 student 283228005 Nov 21 21:17 ERR311258_1.fastq.gz
-rw-r--r--  1 s_11 student 306990611 Nov 21 21:19 ERR311258_2.fastq.gz
-rw-r--r--  1 s_11 student 698057509 Nov 21 21:23 ERR311258.aln.bam
-rw-r--r--  1 s_11 student 668781334 Nov 21 21:24 ERR311258.aln.sort.bam
-rw-r--r--  1 s_11 student     14912 Nov 21 21:24 ERR311258.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  48387545 Nov 21 21:42 ERR311258.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3561 Nov 21 21:42 ERR311258.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763921 Nov 21 21:42 ERR311258.consensus.fa
-rw-r--r--  1 s_11 student 285581691 Nov 21 21:44 ERR311259_1.fastq.gz
-rw-r--r--  1 s_11 student 310955720 Nov 21 21:45 ERR311259_2.fastq.gz
-rw-r--r--  1 s_11 student 704477678 Nov 21 21:49 ERR311259.aln.bam
-rw-r--r--  1 s_11 student 674295131 Nov 21 21:50 ERR311259.aln.sort.bam
-rw-r--r--  1 s_11 student     15000 Nov 21 21:50 ERR311259.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  48841648 Nov 21 22:08 ERR311259.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3484 Nov 21 22:08 ERR311259.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763945 Nov 21 22:09 ERR311259.consensus.fa
-rw-r--r--  1 s_11 student 228562626 Nov 21 22:10 ERR311261_1.fastq.gz
-rw-r--r--  1 s_11 student 248373760 Nov 21 22:11 ERR311261_2.fastq.gz
-rw-r--r--  1 s_11 student 563759197 Nov 21 22:15 ERR311261.aln.bam
-rw-r--r--  1 s_11 student 540946975 Nov 21 22:15 ERR311261.aln.sort.bam
-rw-r--r--  1 s_11 student     14720 Nov 21 22:16 ERR311261.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  46613375 Nov 21 22:30 ERR311261.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3431 Nov 21 22:30 ERR311261.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763927 Nov 21 22:30 ERR311261.consensus.fa
-rw-r--r--  1 s_11 student 229909393 Nov 21 22:31 ERR311262_1.fastq.gz
-rw-r--r--  1 s_11 student 248976399 Nov 21 22:32 ERR311262_2.fastq.gz
-rw-r--r--  1 s_11 student 565985141 Nov 21 22:36 ERR311262.aln.bam
-rw-r--r--  1 s_11 student 542968552 Nov 21 22:37 ERR311262.aln.sort.bam
-rw-r--r--  1 s_11 student     14536 Nov 21 22:37 ERR311262.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  46808565 Nov 21 22:51 ERR311262.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3418 Nov 21 22:51 ERR311262.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763828 Nov 21 22:51 ERR311262.consensus.fa
-rw-r--r--  1 s_11 student 163431228 Nov 21 22:52 ERR338266_1.fastq.gz
-rw-r--r--  1 s_11 student 166045809 Nov 21 22:53 ERR338266_2.fastq.gz
-rw-r--r--  1 s_11 student 385210654 Nov 21 22:55 ERR338266.aln.bam
-rw-r--r--  1 s_11 student 369535527 Nov 21 22:55 ERR338266.aln.sort.bam
-rw-r--r--  1 s_11 student     14328 Nov 21 22:56 ERR338266.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  43764136 Nov 21 23:05 ERR338266.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3486 Nov 21 23:06 ERR338266.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763964 Nov 21 23:06 ERR338266.consensus.fa
-rw-r--r--  1 s_11 student 162568829 Nov 21 23:06 ERR338274_1.fastq.gz
-rw-r--r--  1 s_11 student 165227546 Nov 21 23:07 ERR338274_2.fastq.gz
-rw-r--r--  1 s_11 student 383919037 Nov 21 23:09 ERR338274.aln.bam
-rw-r--r--  1 s_11 student 368305859 Nov 21 23:10 ERR338274.aln.sort.bam
-rw-r--r--  1 s_11 student     14304 Nov 21 23:10 ERR338274.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  43249554 Nov 21 23:20 ERR338274.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3562 Nov 21 23:20 ERR338274.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763961 Nov 21 23:20 ERR338274.consensus.fa
-rw-r--r--  1 s_11 student 141797471 Nov 21 23:21 ERR338282_1.fastq.gz
-rw-r--r--  1 s_11 student 144037790 Nov 21 23:22 ERR338282_2.fastq.gz
-rw-r--r--  1 s_11 student 333749226 Nov 21 23:24 ERR338282.aln.bam
-rw-r--r--  1 s_11 student 320787434 Nov 21 23:24 ERR338282.aln.sort.bam
-rw-r--r--  1 s_11 student     14240 Nov 21 23:24 ERR338282.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  45077602 Nov 21 23:33 ERR338282.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3441 Nov 21 23:33 ERR338282.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763954 Nov 21 23:33 ERR338282.consensus.fa
-rw-r--r--  1 s_11 student 142557099 Nov 21 23:34 ERR338289_1.fastq.gz
-rw-r--r--  1 s_11 student 145020888 Nov 21 23:34 ERR338289_2.fastq.gz
-rw-r--r--  1 s_11 student 336091350 Nov 21 23:36 ERR338289.aln.bam
-rw-r--r--  1 s_11 student 322638299 Nov 21 23:37 ERR338289.aln.sort.bam
-rw-r--r--  1 s_11 student     14192 Nov 21 23:37 ERR338289.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  45188557 Nov 21 23:46 ERR338289.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3416 Nov 21 23:46 ERR338289.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763960 Nov 21 23:46 ERR338289.consensus.fa
-rw-r--r--  1 s_11 student 157346943 Nov 21 23:46 ERR338303_1.fastq.gz
-rw-r--r--  1 s_11 student 160232696 Nov 21 23:47 ERR338303_2.fastq.gz
-rw-r--r--  1 s_11 student 371827151 Nov 21 23:49 ERR338303.aln.bam
-rw-r--r--  1 s_11 student 356486313 Nov 21 23:50 ERR338303.aln.sort.bam
-rw-r--r--  1 s_11 student     14344 Nov 21 23:50 ERR338303.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  44306074 Nov 22 00:00 ERR338303.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3549 Nov 22 00:00 ERR338303.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763966 Nov 22 00:00 ERR338303.consensus.fa
-rw-r--r--  1 s_11 student 143078644 Nov 22 00:01 ERR338306_1.fastq.gz
-rw-r--r--  1 s_11 student 145485360 Nov 22 00:01 ERR338306_2.fastq.gz
-rw-r--r--  1 s_11 student 337128555 Nov 22 00:03 ERR338306.aln.bam
-rw-r--r--  1 s_11 student 323509951 Nov 22 00:04 ERR338306.aln.sort.bam
-rw-r--r--  1 s_11 student     14240 Nov 22 00:04 ERR338306.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  45257453 Nov 22 00:13 ERR338306.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3460 Nov 22 00:13 ERR338306.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763963 Nov 22 00:13 ERR338306.consensus.fa
-rw-r--r--  1 s_11 student 144881617 Nov 22 00:13 ERR338312_1.fastq.gz
-rw-r--r--  1 s_11 student 147502288 Nov 22 00:14 ERR338312_2.fastq.gz
-rw-r--r--  1 s_11 student 342002080 Nov 22 00:16 ERR338312.aln.bam
-rw-r--r--  1 s_11 student 328695724 Nov 22 00:17 ERR338312.aln.sort.bam
-rw-r--r--  1 s_11 student     14240 Nov 22 00:17 ERR338312.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  44656003 Nov 22 00:26 ERR338312.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3418 Nov 22 00:26 ERR338312.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763966 Nov 22 00:26 ERR338312.consensus.fa
-rw-r--r--  1 s_11 student 152111060 Nov 22 00:26 ERR338314_1.fastq.gz
-rw-r--r--  1 s_11 student 154565461 Nov 22 00:27 ERR338314_2.fastq.gz
-rw-r--r--  1 s_11 student 358683782 Nov 22 00:29 ERR338314.aln.bam
-rw-r--r--  1 s_11 student 344416349 Nov 22 00:30 ERR338314.aln.sort.bam
-rw-r--r--  1 s_11 student     14256 Nov 22 00:30 ERR338314.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  44367681 Nov 22 00:39 ERR338314.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3429 Nov 22 00:39 ERR338314.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763966 Nov 22 00:39 ERR338314.consensus.fa
-rw-r--r--  1 s_11 student 174819524 Nov 22 00:41 ERR338258_1.fastq.gz
-rw-r--r--  1 s_11 student 177331812 Nov 22 00:42 ERR338258_2.fastq.gz
-rw-r--r--  1 s_11 student 412320870 Nov 22 00:44 ERR338258.aln.bam
-rw-r--r--  1 s_11 student 395657309 Nov 22 00:45 ERR338258.aln.sort.bam
-rw-r--r--  1 s_11 student     14424 Nov 22 00:45 ERR338258.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  43459146 Nov 22 00:56 ERR338258.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3432 Nov 22 00:56 ERR338258.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763964 Nov 22 00:56 ERR338258.consensus.fa
-rw-r--r--  1 s_11 student 165051432 Nov 22 00:56 ERR338915_1.fastq.gz
-rw-r--r--  1 s_11 student 171655917 Nov 22 00:57 ERR338915_2.fastq.gz
-rw-r--r--  1 s_11 student 394787028 Nov 22 00:59 ERR338915.aln.bam
-rw-r--r--  1 s_11 student 379487002 Nov 22 01:00 ERR338915.aln.sort.bam
-rw-r--r--  1 s_11 student     14336 Nov 22 01:00 ERR338915.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  45102107 Nov 22 01:10 ERR338915.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3526 Nov 22 01:10 ERR338915.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763963 Nov 22 01:10 ERR338915.consensus.fa
-rw-r--r--  1 s_11 student 188051019 Nov 22 01:11 ERR338916_1.fastq.gz
-rw-r--r--  1 s_11 student 196396962 Nov 22 01:12 ERR338916_2.fastq.gz
-rw-r--r--  1 s_11 student 452137696 Nov 22 01:14 ERR338916.aln.bam
-rw-r--r--  1 s_11 student 433582300 Nov 22 01:15 ERR338916.aln.sort.bam
-rw-r--r--  1 s_11 student     14432 Nov 22 01:15 ERR338916.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  44476817 Nov 22 01:26 ERR338916.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3436 Nov 22 01:26 ERR338916.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763946 Nov 22 01:26 ERR338916.consensus.fa
-rw-r--r--  1 s_11 student 195832489 Nov 22 01:27 ERR338923_1.fastq.gz
-rw-r--r--  1 s_11 student 204796409 Nov 22 01:27 ERR338923_2.fastq.gz
-rw-r--r--  1 s_11 student 470771480 Nov 22 01:30 ERR338923.aln.bam
-rw-r--r--  1 s_11 student 451338221 Nov 22 01:31 ERR338923.aln.sort.bam
-rw-r--r--  1 s_11 student     14512 Nov 22 01:31 ERR338923.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  44815273 Nov 22 01:43 ERR338923.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3417 Nov 22 01:43 ERR338923.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763977 Nov 22 01:43 ERR338923.consensus.fa
-rw-r--r--  1 s_11 student 209247007 Nov 22 01:44 ERR338906_1.fastq.gz
-rw-r--r--  1 s_11 student 217539949 Nov 22 01:45 ERR338906_2.fastq.gz
-rw-r--r--  1 s_11 student 501610542 Nov 22 01:48 ERR338906.aln.bam
-rw-r--r--  1 s_11 student 481492086 Nov 22 01:48 ERR338906.aln.sort.bam
-rw-r--r--  1 s_11 student     14408 Nov 22 01:48 ERR338906.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  44909766 Nov 22 02:01 ERR338906.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3465 Nov 22 02:01 ERR338906.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763960 Nov 22 02:02 ERR338906.consensus.fa
-rw-r--r--  1 s_11 student 208548522 Nov 22 02:02 ERR338928_1.fastq.gz
-rw-r--r--  1 s_11 student 217437005 Nov 22 02:03 ERR338928_2.fastq.gz
-rw-r--r--  1 s_11 student 498593253 Nov 22 02:06 ERR338928.aln.bam
-rw-r--r--  1 s_11 student 478756584 Nov 22 02:07 ERR338928.aln.sort.bam
-rw-r--r--  1 s_11 student     14520 Nov 22 02:07 ERR338928.aln.sort.bam.bai
-rw-r--r--  1 s_11 student  44724460 Nov 22 02:19 ERR338928.aln.sort.vcf.gz
-rw-r--r--  1 s_11 student      3437 Nov 22 02:19 ERR338928.aln.sort.vcf.gz.csi
-rw-r--r--  1 s_11 student   4763918 Nov 22 02:20 ERR338928.consensus.fa
-rw-r--r--  1 s_11 student 238197896 Nov 22 02:20 allconsensus.fasta
-rw-r--r--  1 s_11 student 304677830 Nov 22 03:54 allconsensus.phy
drwx------  2 s_11 student       415 Nov 22 11:45 .
-rw-r--r--  1 s_11 student     17313 Nov 27 08:10 RAxML_bootstrap.enteritidis
-rw-r--r--  1 s_11 student      4865 Nov 27 08:10 RAxML_info.enteritidis

Do the timestamps for the two RAxML files indicate that the job is still running correctly but very slowly? Should I cancel the job and only rerun RAxML with six threads?

cbergman commented 6 years ago

yes it looks like raxml is running, but slowly
I would not cancel the job for now
do a wc -l on the RAxML_bootstrap.enteritidis file and see how many bootstrap trees have been generated.

annamtown commented 6 years ago

Looks like only 29 bootstrap trees have been generated:

wc -l RAxML_bootstrap.enteritidis 
29 RAxML_bootstrap.enteritidis

cbergman commented 6 years ago

I would let this RAXML run keep going but it looks like it will take awhile
I would also run fasttree to see what the tree looks like (i.e. are there any samples that are highly divergent) https://wiki.gacrc.uga.edu/wiki/Fasttree
I would then exclude any funny samples based on the fasttree output and launch a new RAXML run with more cores in a different outpt directory

annamtown commented 6 years ago

Here is my fasttree script: c60d597

I've tried to run it three times, but it does not execute properly. This is in my error file:

/var/spool/uge/the_zcluster/compute-13-20/job_scripts/182208: line 10: 10616 Segmentation fault      (core dumped) FastTree -nt -n 10 < /escratch4/s_11/s_11_Aug_17/project/allconsensus.phy > fasttree_file

cbergman commented 6 years ago

I think yu don't need the -n 10 option, try:

FastTree -nt -gtr < /escratch4/s_11/s_11_Aug_17/project/allconsensus.phy > fasttree_file

also have you downloaded your allconsensus.phy and visualized your alignment, eg. in seaview: http://doua.prabi.fr/software/seaview

annamtown commented 6 years ago

I tried the new command and still got an error:

/var/spool/uge/the_zcluster/compute-15-16/job_scripts/183655: line 10: 13469 Segmentation fault      (core dumped) FastTree -nt -gtr < /escratch4/s_11/s_11_Aug_17/project/allconsensus.phy > fasttree_file

Last night, I downloaded seaview and looked at my 10 sample allconsensus.phy file.

cbergman commented 6 years ago

let's look at the fasttree problem in class today
did you see any obviously divergent sequence in allconsensus.phy?
did allconsensus.phy complete with raxml?

annamtown commented 6 years ago

It looks like the first sequence in my alignment has a deletion somewhere around 60,000 bp. Then, the alignment gets a bit out of control...
Yes, it completed RAxML. Here are the files: https://github.com/annamtown/gene8940/blob/master/RAxML_10bestTree.enteritidis https://github.com/annamtown/gene8940/blob/master/RAxML_10bipartitions.enteritidis

annamtown commented 6 years ago

Looks like the addition of the -I flag to remove indels fixed my alignment:

cbergman commented 6 years ago

nice, looking forward to seeing your trees

annamtown commented 6 years ago

Here are the 10 sample trees:

Best tree
Bipartition with bootstrap

My 50-sample run executed well.

I checked the .phy alignment in seaview and the entire alignment looks good:
Best tree:
Bipartition tree:

The outgroup, ERR338264, is the same for both the 10 sample run and then 50 sample run.

annamtown commented 6 years ago

I ran my all sample job this morning and there is still something wrong with a command.

I omitted my curl command to get the meta file from my script. Before I ran the job, I used the curl command via command line so that my file would already be in the directory. I'm assuming that the command to define "i" is the problem because it is not using the ERR#s as the file names for the fastqs.

cbergman commented 6 years ago

I'm not 100% sure I understand. The output from your screen like the first sample (ERR235102) downloaded correctly and is currently being mapped. Why do you think there is a problem?
Also I noticed that the | grep -v ERR526292 is redundant since we removed the pacbio sample in the curl command (| grep -v PacBio)

annamtown commented 6 years ago

The "run_accession" files were not generated from my previous jobs. I'm assuming that the

cut -f6 meta.tvs

command is inserting the header of the column in addition to the ERR#s. Will this affect my allconsensus files?

cbergman commented 6 years ago

you could fix this by changing the for loop:

for i in `cut -f6 meta.tsv | grep -v run_accession`

or you can fix this by changing the cat command:

cat /escratch4/s_11/s_11_Aug_17/allsample/ER*.consensus.fa > /escratch4/s_11/s_11_Aug_17/allsample/allconsensus.fasta

probably the second option is better at this stage so you don't have to run the mapping steps again.

annamtown commented 6 years ago

Great, thank you. I submitted my job to run all samples on Friday and it was still running this afternoon. I just canceled the job and will retry with this new command
I doubt the new all sample job will finish in time to complete my report, so I will write about the 50 sample run I completed last week and discuss the script changes to run all ~700 samples in my conclusion section

annamtown / gene8940

Project Plan #10