Ask away!

Hello,

My workflow seems to be failing as a result of what I think is a samtools error.

Command: minimap2 -t 4 -ax splice -uf -p 1.0 "genome_index.mmi" "seqs.fastq.gz" | samtools view -Sb > "output.bam" 3499 samtools sort -@ 4 "output.bam" -o "WTpoly2_reads_aln_sorted.bam"

Error: [WARNING] Indexing parameters (-k, -w or -H) overridden by parameters used in the prebuilt index. [main_samview] fail to read the header from "-".

I've taken a quick look at the files being maniupulated here (in the offending work directory). The .mmi does not appear to be human readable but is at least not empty (I can provide if you like ... GitHub won't allow me to upload). And seqs.fastq.gz is certainly not empty: 5G compressed and 20G uncompressed (I am pasting just the first entry for your reference below). However, output.bam is empty, and so I think this causes the issue for the samtools sort command. Any idea what could be causing this?

@4aabeb37-0ae5-4289-ab27-e6f246a7e21a runid=215283ee2a07610bc86fc4baad99de671b475048 read=74966 ch=2198 start_time=2023-09-23T00:45:55.980051-04:00 flow_cell_id=PAO05278 protocol_group_id=Vassar_smm092223 sample_id=NB_A11-H11 barcode=barcode84 barcode_alias=barcode84 parent_read_id=4aabeb37-0ae5-4289-ab27-e6f246a7e21a basecall_model_version_id=dna_r10.4.1_e8.2_5khz_400bps_hac@v4.2.0 ATGTTATGTCCTGTACTTCGTTCAGTTACGTATTGCTAAGGTTAAGTAGTGGACCTAGAACCTGTGCCACAGCACCTCGCCTACCGTGACAAGAAAGTTGTCGGTGTCTTTGTGACTGCCTGTCGCTCTATCTTCAGAGGAGAGTCCGCCGCCCGCAAGTTTTTTTTTTTTTTTTTTTTTTTTTTGCCGATGTGATTCTAAAGATGTCATTCATATATTCAAGTTAAAAAATATATAGAATTGATAAGCCATATATAGAAACAATTACTAAAGAGAGAAAAAAAACCAAGTCTGGTAAAAGAAAAAGAAAAAGAGCCTAATATACGACACTTTGGTTCAAATAATTCACCAGAAGTCCTACGTTTTATCCTCCACTTATTATTATATCATCTTGTTAATTTTATTTCTTTAAAATTTAGCAAATTGCTTGTTGGCCTTACCGGCAGCAGCAGAGACCTTGACGACGTTGAATCTAACAGTCTTGGAGATTGGTCTACATTGACCAACGGTAACAATGTCACCAACTTGGACACGGAAAGCTGGGGAGACGTGGACTGGGACGTTCTTGTGTCTCTTTTCGTATCTGTTGTACTTTGGAATGTAATGCAAGTAAGCTCTTTCTGATGACAATGGTACGGTGCATCTTGGTGGAGACGACGGTACCGGTCAAGATCTTACCACGGATGGAAACTAAACCAGTGAATGGACATTTCTTATCAATGTAAGAACCTTCAATAGCGGTCTTTGGGGTCTTGAATCCCAAACCGGCATTCTTATACCATCTCTTGGTTCTCTTGGAAGTCTTGACCTTTGGATTGTTGAAGATGTGAGGTTGCTTTTGGAAAGCTCTTTAGATTGAACAGTTAATTCAGTGGACATCTTTTTTCCCTGGCTTGATACCCCAAAGTCCAAGGGCAATTCGAACTGGAAAGCAATATCAGCACCAACAGAAACACAAAGACACCGACAACTTTCTTGTCACC + %%&'(++)%%(+,++++235891122><=>>FIDDCCFD=714211<BA?BBHGITFFEDDEEEE{HIPCBA@@?@ADJPHH{I{NHE=<<=BCECEIDGHGNHFJHI{{JHPFDICE{F@@AAFBCCBI=888KHFNL{HJEGDDDEC=EHEHJ@D67>?ADRI{{{{{{{{{HBFCIJB?=.,,9{>:9433267998:99:?>E{J{P{;:99@@B<<?{GFKEL{{GICCBFGGHDIHGFEFE{{{{EHN{IHFMHDDB@ACB;{{{{2..AA?@@@@BB??ANNHJJ{G{@?FC;>?DHME=>AAG>???C@?777;>@BW>@?AFQEJSIJHIHFGH{F{QIGEH{{FCCBB^PFHQJJFDIJKLHEGGHKF{{KGDJS{K{GFF{H{OO{I{KKN{HJIKHRMJ{DEJFFDFCBEIGHJG{GFNEEKEHEKLHDDB@=DCCAED?EE;::9?93397>=;:=DCABCDBCEEIIEEABBGKGUEHHJHKJHGABC@BC5546===F6666B@BFJFFDDCEGJEC5;;IFB==?A=76689FEGD@A@AIKGGMFEG@{9=<<BDEKHHIEEGDAA@CFCFEFGED:::9;0-+(&(+(())&0..0022114222+,+.-0,+))1>AC=BKFEGHGCDE{=444--,,02+'('''+-/CDEGIHCA?@EEE{HEFHE{GJ{IF{HPDFF@CA889=B?@COC;988,012677452003404476,***00+((/)6;=::>@FFDEFBFEDHEIG{{{H6666@C.,,,<;FEGDC=77BBFGI8889FEDEDFCBCDEG<;@DLPH@??GG:9:YI{JFFBE7(((788:BBFEFFFDKJE{LHI{GBDDF{@@BDAD?>/../1011.-,-33=>@EELGJEG{HIHIFFBB@ALLGGKFHFFEDHGFHGGIJEDCHCJDEIGFJGHUIFXGHHFEIIOGMFFIGGCC:94&

Hi, can you provide your full input cmd? Are these cdna or direct_rna reads?

Thanks for getting back to me!

These are cDNA reads and I am using sbatch to start the job via slurm. See below:

!/bin/bash

SBATCH -J Aitken_epi2me_poly_noRNA

SBATCH -o Aitken_epi2me_poly_noRNA.out

SBATCH --nodes=1

SBATCH --exclusive

SBATCH --mem=MaxMemPerNode

SBATCH -p emc

cd /work/epi2me

./nextflow -log /work/caitken/cea_nextflow.log run epi2me-labs/wf-transcriptomes -with-trace \ --fastq /work/caitken/data/DegronNanoporeSequencing/Poly \ --ref_genome /work/caitken/data/DegronNanoporeSequencing/sacCer3/20110902_sacCer3.fa \ --ref_annotation /work/caitken/data/DegronNanoporeSequencing/sacCer3/20110902_sacCer3.gff \ --de_analysis \ --sample_sheet /work/caitken/BarcodesPoly.csv \ --out_dir /work/caitken/data/DegronNanoporeSequencing/outputPoly_noRNA \ -c /work/caitken/data/DegronNanoporeSequencing/my_config.cfg

Colin Echeverría Aitken

Assistant Professor Biology Department Biochemistry Program Vassar College @. @.> 845.437.7430

On Nov 16, 2023, at 7:04 AM, Sarah Griffiths @.***> wrote:

Hi, can you provide your full input cmd? Are these cdna or direct_rna reads?

— Reply to this email directly, view it on GitHub https://github.com/epi2me-labs/wf-transcriptomes/issues/45#issuecomment-1814313363, or unsubscribe https://github.com/notifications/unsubscribe-auth/BD2SWUZKMNA5ADBEOP7YCEDYEX6MPAVCNFSM6AAAAAA7M2VUXKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJUGMYTGMZWGM. You are receiving this because you authored the thread.

Hi @cea295933

What is the exit code you are getting from that process that has the error fail to read the header from "-". This step is very memory intensive process and I have seen this same error message/empty bam appear if the alignment process has run out of memory. Ensure the global config has a high memory set and consider adding --minimap2_index_opts to eg. -w 25 or -w 50

You can actually delete this section from any config you have

executor {
    $local {
        cpus = 4
        memory = "8 GB"
    }
}

Thanks for taking the time to look into this!

The error code I see is:

3496 Command executed: 3497 3498 minimap2 -t 4 -ax splice -uf -p 1.0 "genome_index.mmi" "seqs.fastq.gz" | samtools view -Sb > "output.bam" 3499 samtools sort -@ 4 "output.bam" -o "WTpoly2_reads_aln_sorted.bam" 3500 3501 Command exit status: 3502 1 3503 3504 Command output: 3505 (empty) 3506 3507 Command error: 3508 [WARNING] Indexing parameters (-k, -w or -H) overridden by parameters used in the prebuilt index. 3509 [main_samview] fail to read the header from "-". 3510 3511 Work dir: 3512 /work/epi2me/work/85/754a2fa2bd94abf6deaf62d3491903

The contents of my config file are:

1 executor { 2 $local { 3 cpus = 64 4 memory = "256 GB" 5 } 6 }

Our HPC has 512 GB and 128 CPUs available per node. It looks to me like the sbatch code is overriding the config file, because squeue returns 128 CPUs running for the job. But I will try running this with the added code and will amend to config file auto to request all the available CPUs and memory (unless you suggest I try just shy of the max memory, say 480 GB)

Colin Echeverría Aitken

Assistant Professor Biology Department Biochemistry Program Vassar College @. @.> 845.437.7430

On Nov 16, 2023, at 10:42 AM, Sarah Griffiths @.***> wrote:

Hi @cea295933 https://github.com/cea295933 What is the exit code you are getting from that process that has the error fail to read the header from "-". This step is very memory intensive process and I have seen this same error message/empty bam appear if the alignment process has run out of memory. Ensure the global config has a high memory set and consider adding --minimap2_index_opts to eg. -w 25 or -w 50

You can actually delete this section from any config you have

executor { $local { cpus = 4 memory = "8 GB" } } — Reply to this email directly, view it on GitHub https://github.com/epi2me-labs/wf-transcriptomes/issues/45#issuecomment-1814709768, or unsubscribe https://github.com/notifications/unsubscribe-auth/BD2SWUZCD37W2GZWCEQYVM3YEYX5JAVCNFSM6AAAAAA7M2VUXKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJUG4YDSNZWHA. You are receiving this because you were mentioned.

Sorry I see you have supplied the full input cmd above, what is the reference annotation and reference genome you are using if publicly available?

Also how many samples are you running?

I am running two replicates of each sample … I know 3 or more is best but elsewhere on the GitHub I see that 2 is ok (and know that DESeq2, for example, can accept just 2 replicates. This is a historical set of samples that we used for Illumina sequencing and so don’t have more replicates. I’m trying to compare our results here to what we obtained previously using DESeq2 on those Illumina runs to determine whether to purchase a PromethION instrument.

We actually have 8 samples total, but I am running them in two batches. They represent either total RNA or a fraction of Toal RNA (associated with ribosomes). We have two replicates for each condition (WT/Mut and Total/Poly). All I really need is a read counts so that I can go into DESeq2 to repeat our previous analysis and compare. I’m certainly interested in the DE numbers the workflow provides, but I also want to determine DE for the (Poly/Total) reads, when comparing mutant to WT, and I know how to setup DESeq2 to achieve that using the R package directly.

Thanks!

Colin Echeverría Aitken

Assistant Professor Biology Department Biochemistry Program Vassar College @. @.> 845.437.7430

On Nov 16, 2023, at 12:13 PM, Sarah Griffiths @.***> wrote:

Also how many samples are you running?

— Reply to this email directly, view it on GitHub https://github.com/epi2me-labs/wf-transcriptomes/issues/45#issuecomment-1814880640, or unsubscribe https://github.com/notifications/unsubscribe-auth/BD2SWU2AGQOY5HP4HBBSW2LYEZCRZAVCNFSM6AAAAAA7M2VUXKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJUHA4DANRUGA. You are receiving this because you were mentioned.

If you have a reference transcriptome available and supply it to the ref_transcriptome parameter does the workflow progress past this part of the workflow? Trying to work out if the reads or something to do with the string tie merged transcriptome we are using could be causing this error.

If I supply a reference guided transcritptome, it crashes at the salmon step … I get an error about needing to upgrade salmon (described in issue 43 but also pasted below):

Process pipeline:differential_expression:count_transcripts (1) terminated with an error exit status (1)

Command executed:

salmon quant --noErrorModel -p "4" -t "ammended.ref_transcriptome" -l SF -a "WTpoly1_reads_aln_sorted.bam" -o counts mv counts/quant.sf "WTpoly1.transcript_counts.tsv" seqkit bam "WTpoly1_reads_aln_sorted.bam" 2> "WTpoly1.seqkit.stats"

Command exit status: 1

Command output: (empty)

Command error: Version Info: ### PLEASE UPGRADE SALMON ###

Colin Echeverría Aitken

Assistant Professor Biology Department Biochemistry Program Vassar College @. @.> 845.437.7430

On Nov 16, 2023, at 12:25 PM, Sarah Griffiths @.***> wrote:

If you have a reference transcriptome available and supply it to the ref_transcriptome parameter does the workflow progress past this part of the workflow? Trying to work out if the reads or something to do with the string tie merged transcriptome we are using could be causing this error.

— Reply to this email directly, view it on GitHub https://github.com/epi2me-labs/wf-transcriptomes/issues/45#issuecomment-1814905735, or unsubscribe https://github.com/notifications/unsubscribe-auth/BD2SWU6IKGZIVUYMHKYMDILYEZEAZAVCNFSM6AAAAAA7M2VUXKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJUHEYDKNZTGU. You are receiving this because you were mentioned.

Could you try decreasing the memory directive to something like 64GB? Is there any further message in the salmon error. I think the part about updating the version isn't what's causing it to break - As we have that message in our automated tests as well which don't fail.

A few questions:

You want me to decrease memory now? Via sbatch or config file, or both?
Do you want me to run with transcriptome to produce salmon error? I can also provide those log files.
what is the syntax of the minimap2_index_opts command? When I enter "minimap2_index_opts -w 25" within my sbatch file I get an error.

Here's what I get if I run DE only a la GitHub test code (DE only test code from GitHub runs fine):

613 ERROR ~ Error executing process > 'pipeline:differential_expression:count_transcripts (1)' 614 615 Caused by: 616 Process pipeline:differential_expression:count_transcripts (1) terminated with an error exit status (1) 617 618 Command executed: 619 620 salmon quant --noErrorModel -p "4" -t "ammended.ref_transcriptome" -l SF -a "WTpoly1_reads_aln_sorted.bam" -o counts 621 mv counts/quant.sf "WTpoly1.transcript_counts.tsv" 622 seqkit bam "WTpoly1_reads_aln_sorted.bam" 2> "WTpoly1.seqkit.stats" 623 624 Command exit status: 625 1 626 627 Command output: 628 (empty) 629 630 Command error: 631 Version Info: ### PLEASE UPGRADE SALMON ### 632 ### A newer version of salmon with important bug fixes and improvements is available. #### 633 ### 634 The newest version, available at https://github.com/COMBINE-lab/salmon/releases 635 contains new features, improvements, and bug fixes; please upgrade at your 636 earliest convenience. 637 ### 638 Sign up for the salmon mailing list to hear about new versions, features and updates at: 639 https://oceangenomics.com/subscribe 640 # salmon (alignment-based) v1.9.0 641 # [ program ] => salmon 642 # [ command ] => quant 643 # [ noErrorModel ] => { } 644 # [ threads ] => { 4 } 645 # [ targets ] => { ammended.ref_transcriptome } 646 # [ libType ] => { SF } 647 # [ alignments ] => { WTpoly1_reads_aln_sorted.bam } 648 # [ output ] => { counts } 649 Logs will be written to counts/logs 650 [2023-11-14 22:10:17.514] [jointLog] [info] setting maxHashResizeThreads to 4 651 [2023-11-14 22:10:17.514] [jointLog] [info] Fragment incompatibility prior below threshold. Incompatible fragments will be ignored. 652 Library format { type:single end, relative orientation:none, strandedness:sense } 653 [2023-11-14 22:10:17.514] [jointLog] [info] numQuantThreads = 2 654 parseThreads = 2 655 Checking that provided alignment files have consistent headers . . . done 656 Populating targets from aln = "WTpoly1_reads_aln_sorted.bam", fasta = "ammended. ref_transcriptome" . . .done

If I run without DE, no issues with these data or with test data.

One issue I can see with running with supplied transcriptome is that there are no gold-standard (SGD, RefSeq, NCBI, etc.) yeast transcriptomes that contain good UTR info. In fact, most (if not all) are just ORFs, and just one isoform per gene. There are studies that have tried to investigate this(Pelechano, et al. for example) but they do not report a curated transcriptome and these have not been adopted by the big databases.

Just to clarify:

Thanks so much for your help!
Happy to run whatever you need ... not trying to avoid that by supplying previous logs. Just let me know what you'd like to test. Our HPC is currently pretty wide open so I can run multiple jobs at a time.

yes sorry try either decreasing in the executor or just delete the executor

1 executor {
2     $local {
3         cpus = 64
4         memory = "32 GB"
5     }
6 }

Yes please, if you use this test in your environment do you get the salmon error?

wget -O differential_expression.tar.gz https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-isoforms/differential_expression.tar.gz && tar -xzvf differential_expression.tar.gz
nextflow run epi2me-labs/wf-transcriptomes \
--fastq  differential_expression/differential_expression_fastq \
--transcriptome-source precomputed \
--de_analysis \
--ref_genome differential_expression/hg38_chr20.fa \
--ref_annotation differential_expression/gencode.v22.annotation.chr20.gff3 \
--direct_rna --minimap2_index_opts '-k 15' \
--ref_transcriptome differential_expression/ref_transcriptome.fasta \
--sample_sheet test_data/sample_sheet.csv

Put quotes around the index opts. --minimap2_index_opts "-w 25"

Test DE analysis runs fine. Log below:

Last login: Wed Nov 15 19:06:08 on console

The default interactive shell is now zsh. To update your account to use zsh, please run chsh -s /bin/zsh. For more details, please visit https://support.apple.com/kb/HT208050. caitken:~ caitken$ ssh caitken@jr.vassar.edu ^C caitken:~ caitken$ ssh caitken@jr.vassar.edu caitken@jr.vassar.edu's password:

Welcome to Ubuntu 20.04.6 LTS (GNU/Linux 5.4.0-153-generic x86_64)

/ / / / __ / // / \/ \/ \/ _ \/ _/ / _ / // / // / // / / /
// //_/ ./ ./\//
// /_/

System information as of Thu 16 Nov 2023 12:29:58 PM EST

System load: 0.0 Users logged in: 3 Usage of /: 17.4% of 1.79TB IPv4 address for docker0: 172.17.0.1 Memory usage: 17% IPv4 address for eth0: 192.168.0.5 Swap usage: 0% IPv4 address for eth1: 143.229.7.20 Processes: 608 Last login: Thu Nov 16 10:55:33 2023 from 143.229.41.79 caitken@ccas020:~[12:29]$ squeue JOBID PARTITION NAME USER ST TIME NODES CPUS NODELIST(REASON) caitken@ccas020:~[12:30]$ cd /work/epi2me/ caitken@ccas020:/work/epi2me[12:30]$ ls 50 nextflow Aitken_epi2me_20231110_poly_emc_noDE.out outdir Aitken_epi2me_20231110_total_emc2.out output Aitken_epi2me_20231110_total_emcK10.out Poly_epi2me_noRNA_sbatch Aitken_epi2me_20231110_total_emcK11.out Poly_epi2me_sbatch_emc_DEonly Aitken_epi2me_20231110_total_emcK14.out Poly_epi2me_sbatch_emc_noDE.sh Aitken_epi2me_20231110_total_emc_noDE.out test_data.tar.gz Aitken_epi2me_20231110_total_emc.out testDE_sbatch Aitken_epi2me_poly_emc_DEonly.out Total_epi2me_noRNA_sbatch Aitken_epi2me_poly_noRNA2.out Total_epi2me_sbatch_emc Aitken_epi2me_poly_noRNA.out Total_epi2me_sbatch_emc_DEonly Aitken_epi2me_testDE.out Total_epi2me_sbatch_emc_K10 Aitken_epi2me_total_emc_DEonly.out Total_epi2me_sbatch_emc_K11 Aitken_epi2me_total_noRNA.out Total_epi2me_sbatch_emc_K14.sh chr20 Total_epi2me_sbatch_emc_NCBI.sh differential_expression Total_epi2me_sbatch_emc_noDE differential_expression.tar.gz work ERR6053095_chr20.fastq workspace_dir caitken@ccas020:/work/epi2me[12:30]$ vi Aitken_epi2me_poly_noRNA2.out caitken@ccas020:/work/epi2me[12:30]$ vi Poly_epi2me_noRNA_sbatch caitken@ccas020:/work/epi2me[12:35]$ vi Aitken_epi2me_poly_noRNA2.out caitken@ccas020:/work/epi2me[12:36]$ sbatch Poly_epi2me_noRNA_sbatch Submitted batch job 36004 caitken@ccas020:/work/epi2me[12:37]$ squeue JOBID PARTITION NAME USER ST TIME NODES CPUS NODELIST(REASON) caitken@ccas020:/work/epi2me[12:37]$ squeue JOBID PARTITION NAME USER ST TIME NODES CPUS NODELIST(REASON) 36005 general augcccation fgoggins R 0:19 1 10 node1 caitken@ccas020:/work/epi2me[12:37]$ vi Aitken_epi2me_poly_noRNA2.out caitken@ccas020:/work/epi2me[12:38]$ sbatch Poly_epi2me_noRNA_sbatch Submitted batch job 36007 caitken@ccas020:/work/epi2me[12:38]$ squeue JOBID PARTITION NAME USER ST TIME NODES CPUS NODELIST(REASON) 36006 general 631G fgoggins R 0:16 1 10 node1 36005 general augcccation fgoggins R 0:49 1 10 node1 36007 emc Aitken_epi2me_po caitken R 0:01 1 128 node5 caitken@ccas020:/work/epi2me[12:38]$ squeue JOBID PARTITION NAME USER ST TIME NODES CPUS NODELIST(REASON) 36006 general 631G fgoggins R 0:21 1 10 node1 36005 general augcccation fgoggins R 0:54 1 10 node1 caitken@ccas020:/work/epi2me[12:38]$ vi Aitken_epi2me_poly_noRNA2.out caitken@ccas020:/work/epi2me[12:39]$ sbatch Poly_epi2me_noRNA_sbatch Submitted batch job 36008 caitken@ccas020:/work/epi2me[12:39]$ squeue JOBID PARTITION NAME USER ST TIME NODES CPUS NODELIST(REASON) 36006 general 631G fgoggins R 1:04 1 10 node1 36005 general augcccation fgoggins R 1:37 1 10 node1 36008 emc Aitken_epi2me_po caitken R 0:01 1 128 node5 caitken@ccas020:/work/epi2me[12:39]$ squeue JOBID PARTITION NAME USER ST TIME NODES CPUS NODELIST(REASON) 36006 general 631G fgoggins R 1:08 1 10 node1 36005 general augcccation fgoggins R 1:41 1 10 node1 36008 emc Aitken_epi2me_po caitken R 0:05 1 128 node5 caitken@ccas020:/work/epi2me[12:39]$ squeue JOBID PARTITION NAME USER ST TIME NODES CPUS NODELIST(REASON) 36006 general 631G fgoggins R 1:10 1 10 node1 36005 general augcccation fgoggins R 1:43 1 10 node1 36008 emc Aitken_epi2me_po caitken R 0:07 1 128 node5 caitken@ccas020:/work/epi2me[12:39]$ squeue JOBID PARTITION NAME USER ST TIME NODES CPUS NODELIST(REASON) 36006 general 631G fgoggins R 1:13 1 10 node1 36005 general augcccation fgoggins R 1:46 1 10 node1 36008 emc Aitken_epi2me_po caitken R 0:10 1 128 node5 caitken@ccas020:/work/epi2me[12:39]$ squeue JOBID PARTITION NAME USER ST TIME NODES CPUS NODELIST(REASON) 36006 general 631G fgoggins R 1:20 1 10 node1 36005 general augcccation fgoggins R 1:53 1 10 node1 caitken@ccas020:/work/epi2me[12:39]$ vi Aitken_epi2me_poly_noRNA2.out caitken@ccas020:/work/epi2me[12:40]$ sbatch Poly_epi2me_noRNA_sbatch Submitted batch job 36009 caitken@ccas020:/work/epi2me[12:40]$ squeue JOBID PARTITION NAME USER ST TIME NODES CPUS NODELIST(REASON) 36006 general 631G fgoggins R 2:33 1 10 node1 36009 emc Aitken_epi2me_po caitken R 0:01 1 128 node5 caitken@ccas020:/work/epi2me[12:40]$ squeue JOBID PARTITION NAME USER ST TIME NODES CPUS NODELIST(REASON) 36006 general 631G fgoggins R 2:35 1 10 node1 36009 emc Aitken_epi2me_po caitken R 0:03 1 128 node5 caitken@ccas020:/work/epi2me[12:40]$ squeue JOBID PARTITION NAME USER ST TIME NODES CPUS NODELIST(REASON) 36006 general 631G fgoggins R 2:36 1 10 node1 36009 emc Aitken_epi2me_po caitken R 0:04 1 128 node5 caitken@ccas020:/work/epi2me[12:40]$ squeue JOBID PARTITION NAME USER ST TIME NODES CPUS NODELIST(REASON) 36006 general 631G fgoggins R 2:38 1 10 node1 36009 emc Aitken_epi2me_po caitken R 0:06 1 128 node5 caitken@ccas020:/work/epi2me[12:40]$ squeue JOBID PARTITION NAME USER ST TIME NODES CPUS NODELIST(REASON) 36006 general 631G fgoggins R 2:40 1 10 node1 36009 emc Aitken_epi2me_po caitken R 0:08 1 128 node5 caitken@ccas020:/work/epi2me[12:40]$ squeue JOBID PARTITION NAME USER ST TIME NODES CPUS NODELIST(REASON) caitken@ccas020:/work/epi2me[12:40]$ vi Aitken_epi2me_poly_noRNA2.out caitken@ccas020:/work/epi2me[12:43]$ ls 25 nextflow 50 outdir Aitken_epi2me_20231110_poly_emc_noDE.out output Aitken_epi2me_20231110_total_emc2.out Poly_epi2me_noRNA_sbatch Aitken_epi2me_20231110_total_emcK10.out Poly_epi2me_sbatch_emc_DEonly Aitken_epi2me_20231110_total_emcK11.out Poly_epi2me_sbatch_emc_noDE.sh Aitken_epi2me_20231110_total_emcK14.out test_data.tar.gz Aitken_epi2me_20231110_total_emc_noDE.out testDE_sbatch Aitken_epi2me_20231110_total_emc.out Total_epi2me_noRNA_sbatch Aitken_epi2me_poly_emc_DEonly.out Total_epi2me_sbatch_emc Aitken_epi2me_poly_noRNA2.out Total_epi2me_sbatch_emc_DEonly Aitken_epi2me_poly_noRNA.out Total_epi2me_sbatch_emc_K10 Aitken_epi2me_testDE.out Total_epi2me_sbatch_emc_K11 Aitken_epi2me_total_emc_DEonly.out Total_epi2me_sbatch_emc_K14.sh Aitken_epi2me_total_noRNA.out Total_epi2me_sbatch_emc_NCBI.sh chr20 Total_epi2me_sbatch_emc_noDE differential_expression work differential_expression.tar.gz workspace_dir ERR6053095_chr20.fastq caitken@ccas020:/work/epi2me[12:43]$ vi Aitken_epi2me_poly_emc_DEonly.out caitken@ccas020:/work/epi2me[12:46]$ vi Aitken_epi2me_20231110_poly_emc_noDE.out caitken@ccas020:/work/epi2me[12:47]$ vi Aitken_epi2me_total_emc_DEonly.out caitken@ccas020:/work/epi2me[12:56]$ vi Aitken_epi2me_testDE.out

24014 [ee/9bb70f] process > pipeline:differential_expre... [100%] 1 of 1 ✔ 24015 [06/6fef76] process > pipeline:differential_expre... [100%] 6 of 6 ✔ 24016 [5a/06e947] process > pipeline:differential_expre... [100%] 6 of 6 ✔ 24017 [f3/3d0571] process > pipeline:differential_expre... [100%] 1 of 1 ✔ 24018 [29/44dba5] process > pipeline:differential_expre... [100%] 1 of 1 ✔ 24019 [88/6a0146] process > pipeline:differential_expre... [100%] 1 of 1 ✔ 24020 [d2/edc948] process > pipeline:differential_expre... [100%] 1 of 1 ✔ 24021 [87/d5752d] process > pipeline:makeReport (1) [100%] 1 of 1 ✔ 24022 [59/5d6d71] process > output (22) [ 90%] 20 of 22 24023 24024 executor > local (58) 24025 [25/87e83a] process > validate_sample_sheet [100%] 1 of 1 ✔ 24026 [cd/a8bb45] process > fastcat (5) [100%] 6 of 6 ✔ 24027 [e2/6e936a] process > pipeline:preprocess_ref_ann... [100%] 1 of 1 ✔ 24028 [c7/2a7592] process > pipeline:collectFastqIngres... [100%] 6 of 6 ✔ 24029 [88/616c92] process > pipeline:getVersions [100%] 1 of 1 ✔ 24030 [0e/9b4ea0] process > pipeline:getParams [100%] 1 of 1 ✔ 24031 [b8/e04e62] process > pipeline:preprocess_ref_tra... [100%] 1 of 1 ✔ 24032 [b1/35adc2] process > pipeline:differential_expre... [100%] 1 of 1 ✔ 24033 [ee/9bb70f] process > pipeline:differential_expre... [100%] 1 of 1 ✔ 24034 [06/6fef76] process > pipeline:differential_expre... [100%] 6 of 6 ✔ 24035 [5a/06e947] process > pipeline:differential_expre... [100%] 6 of 6 ✔ 24036 [f3/3d0571] process > pipeline:differential_expre... [100%] 1 of 1 ✔ 24037 [29/44dba5] process > pipeline:differential_expre... [100%] 1 of 1 ✔ 24038 [88/6a0146] process > pipeline:differential_expre... [100%] 1 of 1 ✔ 24039 [d2/edc948] process > pipeline:differential_expre... [100%] 1 of 1 ✔ 24040 [87/d5752d] process > pipeline:makeReport (1) [100%] 1 of 1 ✔ 24041 [bf/2e04ad] process > output (21) [100%] 22 of 22 ✔ 24042 24043 executor > local (58) 24044 [25/87e83a] process > validate_sample_sheet [100%] 1 of 1 ✔ 24045 [cd/a8bb45] process > fastcat (5) [100%] 6 of 6 ✔ 24046 [e2/6e936a] process > pipeline:preprocess_ref_ann... [100%] 1 of 1 ✔ 24047 [c7/2a7592] process > pipeline:collectFastqIngres... [100%] 6 of 6 ✔ 24048 [88/616c92] process > pipeline:getVersions [100%] 1 of 1 ✔ 24049 [0e/9b4ea0] process > pipeline:getParams [100%] 1 of 1 ✔ 24050 [b8/e04e62] process > pipeline:preprocess_ref_tra... [100%] 1 of 1 ✔ 24051 [b1/35adc2] process > pipeline:differential_expre... [100%] 1 of 1 ✔ 24052 [ee/9bb70f] process > pipeline:differential_expre... [100%] 1 of 1 ✔ 24053 [06/6fef76] process > pipeline:differential_expre... [100%] 6 of 6 ✔ 24054 [5a/06e947] process > pipeline:differential_expre... [100%] 6 of 6 ✔ 24055 [f3/3d0571] process > pipeline:differential_expre... [100%] 1 of 1 ✔ 24056 [29/44dba5] process > pipeline:differential_expre... [100%] 1 of 1 ✔ 24057 [88/6a0146] process > pipeline:differential_expre... [100%] 1 of 1 ✔ 24058 [d2/edc948] process > pipeline:differential_expre... [100%] 1 of 1 ✔ 24059 [87/d5752d] process > pipeline:makeReport (1) [100%] 1 of 1 ✔ 24060 [bf/2e04ad] process > output (21) [100%] 22 of 22 ✔ 24061 Completed at: 14-Nov-2023 10:58:58 24062 Duration : 2m 55s 24063 CPU hours : 0.1 24064 Succeeded : 58 24065

ok and now try reference guided

nextflow run epi2me-labs/wf-transcriptomes \
--fastq  differential_expression/differential_expression_fastq \
--de_analysis \
--ref_genome differential_expression/hg38_chr20.fa --transcriptome-source reference-guided \
--ref_annotation differential_expression/gencode.v22.annotation.chr20.gtf \
--direct_rna --minimap2_index_opts '-k 15'  --sample_sheet test_data/sample_sheet.csv

running all three now ... btw, the --sampel_sheet test_data/sample_sheet.csv gave me problems previously ... I don't remember if it's not included in the test data one downloads, or whether it's in a different directory or what ... I ended up having to create my own sample sheet (or move it to the directory ... can't remember)

... perhaps you could look into that separately... it might be giving others trouble too

good idea, yes will add it to the downloadable test set

two jobs with my data appear to be running stably, and test run ran without issues (see below). when I get samtools error, it is after long stable run (over 1 h). whereas when I get salmon error, it's usually after 10 minutes of run or less. Both running now for at least 10 m

N E X T F L O W ~ version 23.10.0 Launching https://github.com/epi2me-labs/wf-transcriptomes [boring_planck] DSL2 - revision: 0a8fe19e7c [master]

WARN: Found unexpected parameters:

--transcriptome-source: reference-guided
Ignore this warning: params.schema_ignore_params = "transcriptome-source"

|||||||||| _ __ _ ** |||||||||| | __| _ | | \/ | __| | | | | ||||| | | | |) | | ) | |\/| | _| ___| |/ ` | ' \/ | ||||| | |_| /| | / /| | | | |_|| | (| | |) _ \ |||||||||| |____|_| |_|___|| ||| ||\,|._/|/ |||||||||| wf-transcriptomes v0.4.1-g0a8fe19

Core Nextflow options revision : master runName : boring_planck containerEngine : docker container : [withLabel:isoforms:ontresearch/wf-transcriptomes:shae7c9f184996a384e99be68e790f0612f0c732867, withLabel:wf_common:ontresearch/wf-common:sha0a6dc21fac17291f4acb2e0f67bcdec7bf63e6b7] launchDir : /work/epi2me workDir : /work/epi2me/work projectDir : /home/caitken/.nextflow/assets/epi2me-labs/wf-transcriptomes userName : caitken profile : standard configFiles : /home/caitken/.nextflow/assets/epi2me-labs/wf-transcriptomes/nextflow.config

Input Options fastq : differential_expression/differential_expression_fastq ref_genome : differential_expression/hg38_chr20.fa ref_annotation : differential_expression/gencode.v22.annotation.chr20.gtf direct_rna : true

Sample Options sample_sheet : differential_expression/sample_sheet.csv

Options for reference-based workflow minimap2_index_opts: -k 15

Differential Expression Options de_analysis : true

!! Only displaying parameters that differ from the pipeline defaults !!

If you use epi2me-labs/wf-transcriptomes for your analysis please cite:

The nf-core framework https://doi.org/10.1038/s41587-020-0439-x

This is epi2me-labs/wf-transcriptomes v0.4.1-g0a8fe19.

Checking fastq input. executor > local (119) [1b/620c1e] process > validate_sample_sheet [100%] 1 of 1 ✔ [91/df2f5c] process > fastcat (6) [100%] 6 of 6 ✔ [a2/4fc9fc] process > pipeline:preprocess_ref_annotation [100%] 1 of 1 ✔ [46/5e2ffe] process > pipeline:collectFastqIngressResultsInDir (6) [100%] 6 of 6 ✔ [90/bb2a11] process > pipeline:getVersions [100%] 1 of 1 ✔ [1e/02f132] process > pipeline:getParams [100%] 1 of 1 ✔ [c4/1f960d] process > pipeline:build_minimap_index (1) [100%] 1 of 1 ✔ [d8/c25222] process > pipeline:reference_assembly:map_reads (6) [100%] 6 of 6 ✔ [90/7a7229] process > pipeline:split_bam (6) [100%] 6 of 6 ✔ [29/517ddb] process > pipeline:assemble_transcripts (6) [100%] 6 of 6 ✔ [80/d44317] process > pipeline:merge_gff_bundles (6) [100%] 6 of 6 ✔ [2f/7955ff] process > pipeline:run_gffcompare (6) [100%] 6 of 6 ✔ [b9/bc2600] process > pipeline:get_transcriptome (6) [100%] 6 of 6 ✔ [2a/5f66bf] process > pipeline:merge_transcriptomes (1) [100%] 1 of 1 ✔ [bf/9fd364] process > pipeline:differential_expression:checkSampleSheetCond... [100%] 1 of 1 ✔ [11/844a2e] process > pipeline:differential_expression:build_minimapindex... [100%] 1 of 1 ✔ [06/51b1ab] process > pipeline:differential_expression:map_transcriptome (2) [100%] 6 of 6 ✔ [d3/469d56] process > pipeline:differential_expression:count_transcripts (6) [100%] 6 of 6 ✔ [3b/4f7573] process > pipeline:differential_expression:mergeCounts [100%] 1 of 1 ✔ [70/2b6988] process > pipeline:differential_expression:mergeTPM [100%] 1 of 1 ✔ [61/5666c7] process > pipeline:differential_expression:deAnalysis (1) [100%] 1 of 1 ✔ [5b/9a1800] process > pipeline:differential_expression:plotResults (1) [100%] 1 of 1 ✔ [2a/cb76df] process > pipeline:makeReport (1) [100%] 1 of 1 ✔ [ad/9cda77] process > output (42) [100%] 46 of 46 ✔ Completed at: 16-Nov-2023 13:10:44 Duration : 5m 7s CPU hours : 0.2 Succeeded : 119

circle back once those have completed?

Thanks so much for all of your help!

No worries, sorry it's not working! For the salmon breakage one - if you go in to a work directory for the map_transcriptome process do the bams contain alignments? I was able to recreate the error you had in the other ticket by inputting an empty bam to that step. That's an error we should be catching and reporting to the user.

Perhaps if you run the workflow with your inputs without the --de_analysis parameter you could share the output report wf-transcriptome-report.html just to check there are alignments between the reference_genome and your reads.

BAM file doesn’t look empty … scp to my local machine is 40% at 762 MB transferred

du -sh only show 4.0K, however.

Colin Echeverría Aitken

Assistant Professor Biology Department Biochemistry Program Vassar College @. @.> 845.437.7430

On Nov 16, 2023, at 1:27 PM, Sarah Griffiths @.***> wrote:

No worries, sorry it's not working! For the salmon breakage one - if you go in to a work directory for the map_transcriptome process do the bams contain alignments? I was able to recreate the error you had in the other ticket by inputting an empty bam to that step. That's an error we should be catching and reporting to the user. Perhaps if you run the workflow with your inputs without the --de_analysis parameter you could share the output report wf-transcriptome-report.html just to check there are alignments between the reference_genome and your reads.

— Reply to this email directly, view it on GitHub https://github.com/epi2me-labs/wf-transcriptomes/issues/45#issuecomment-1815007539, or unsubscribe https://github.com/notifications/unsubscribe-auth/BD2SWUYE5ZNV6JLPIKYZZ53YEZLKZAVCNFSM6AAAAAA7M2VUXKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJVGAYDONJTHE. You are receiving this because you were mentioned.

tried to send .html reports but I can't attach in GitHub

I think if you zip them then you can attach

I have a meeting until later this afternoon ... so not ignoring if I'm radio silent. Will send along results of current runs once I have them. I also added a third run with the reference-guided (with our data) as a point of comparison

From the reports your data looks great, How large is the .mmi file in the failed minimap/samtools process.

I managed to create this error by not providing enough memory to this process. And it results in an empty bam. I wonder if when you had the memory directive set to 256GB and it was running 4 processes in parallel it was causing out of memory. You could also try just inputting one sample at a time to see if it still breaks - just to confirm its not a memory issue.

Caused by:
  Process `pipeline:differential_expression:map_transcriptome (6)` terminated with an error exit status (1)

Command executed:

  minimap2 -t 4 -ax splice -uf -p 1.0 "genome_index.mmi" "seqs.fastq.gz"     | samtools view -Sb > "output.bam"
  samtools sort -@ 4 "output.bam" -o "sample04_reads_aln_sorted.bam"

Command exit status:
  1

Command output:
  (empty)

Command error:
  [main_samview] fail to read the header from "-".

Interesting … the question I have about that approach is that it already works without the DE analysis. And I need to input more samples to get the DE analysis to work, right? I suppose the other possibility would be to take a subset (just some of the fastq files with reads) of the four samples (2 conditions, 2 replicates).

Colin Echeverría Aitken

Assistant Professor Biology Department Biochemistry Program Vassar College @. @.> 845.437.7430

On Nov 16, 2023, at 2:51 PM, Sarah Griffiths @.***> wrote:

From the reports your data looks great, How large is the .mmi file in the failed minimap/samtools process.

I managed to create this error by not providing enough memory to this process. And it results in an empty bam. I wonder if when you had the memory directive set to 256GB and it was running 4 processes in parallel it was causing out of memory. You could also try just inputting one sample at a time to see if it still breaks - just to confirm its not a memory issue.

Caused by: Process pipeline:differential_expression:map_transcriptome (6) terminated with an error exit status (1)

Command executed:

minimap2 -t 4 -ax splice -uf -p 1.0 "genome_index.mmi" "seqs.fastq.gz" | samtools view -Sb > "output.bam" samtools sort -@ 4 "output.bam" -o "sample04_reads_aln_sorted.bam"

Command exit status: 1

Command output: (empty)

Command error: [main_samview] fail to read the header from "-". — Reply to this email directly, view it on GitHub https://github.com/epi2me-labs/wf-transcriptomes/issues/45#issuecomment-1815213081, or unsubscribe https://github.com/notifications/unsubscribe-auth/BD2SWUZL2UBWT2MY52PIIGTYEZVCZAVCNFSM6AAAAAA7M2VUXKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJVGIYTGMBYGE. You are receiving this because you were mentioned.

Perhaps this is another way of testing it: I’ve sent the same job as previously (no reference transcriptome but with DE analysis) and assigned it to two nodes, giving it access to 256 CPU and 1024 GB memory.

Do you know of the best way to check on its memory usage? When I use squeue or sacct, I don’t get that info.

Colin Echeverría Aitken

Assistant Professor Biology Department Biochemistry Program Vassar College @. @.> 845.437.7430

On Nov 16, 2023, at 5:31 PM, Colin Aitken @.***> wrote:

Interesting … the question I have about that approach is that it already works without the DE analysis. And I need to input more samples to get the DE analysis to work, right? I suppose the other possibility would be to take a subset (just some of the fastq files with reads) of the four samples (2 conditions, 2 replicates).

Colin Echeverría Aitken

Assistant Professor Biology Department Biochemistry Program Vassar College @. @.> 845.437.7430

On Nov 16, 2023, at 2:51 PM, Sarah Griffiths @.***> wrote:

From the reports your data looks great, How large is the .mmi file in the failed minimap/samtools process.

I managed to create this error by not providing enough memory to this process. And it results in an empty bam. I wonder if when you had the memory directive set to 256GB and it was running 4 processes in parallel it was causing out of memory. You could also try just inputting one sample at a time to see if it still breaks - just to confirm its not a memory issue.

Caused by: Process pipeline:differential_expression:map_transcriptome (6) terminated with an error exit status (1)

Command executed:

minimap2 -t 4 -ax splice -uf -p 1.0 "genome_index.mmi" "seqs.fastq.gz" | samtools view -Sb > "output.bam" samtools sort -@ 4 "output.bam" -o "sample04_reads_aln_sorted.bam"

Command exit status: 1

Command output: (empty)

Command error: [main_samview] fail to read the header from "-". — Reply to this email directly, view it on GitHub https://github.com/epi2me-labs/wf-transcriptomes/issues/45#issuecomment-1815213081, or unsubscribe https://github.com/notifications/unsubscribe-auth/BD2SWUZL2UBWT2MY52PIIGTYEZVCZAVCNFSM6AAAAAA7M2VUXKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJVGIYTGMBYGE. You are receiving this because you were mentioned.

Index (.mmi) file is approximately 30 MB, I believe

Here is the memory usage for the jobs I’ve run today:

AveCPU     AveRSS    CPUTime ExitCode     MaxRSS      NCPUS   NNodes

                    00:21:20    127:0                   128        1

00:00:00 1084K 00:21:20 127:0 1084K 128 1 00:23:28 1:0 128 1 00:00:00 1076K 00:23:28 1:0 1076K 128 1 00:04:16 127:0 128 1 00:00:00 1076K 00:04:16 127:0 1076K 128 1 00:04:16 1:0 128 1 00:00:00 1076K 00:04:16 1:0 1076K 128 1 00:34:08 1:0 128 1 00:00:00 1076K 00:34:08 1:0 1076K 128 1 00:19:12 1:0 128 1 00:00:00 1080K 00:19:12 1:0 1080K 128 1 5-22:19:44 1:0 128 1 00:02:15 2210644K 5-22:19:44 1:0 2705508K 128 1 5-23:51:28 1:0 128 1 00:02:12 2574452K 5-23:51:28 1:0 2574452K 128 1 00:00:00 5608K 04:35:12 0:0 5608K 128 1 00:00:00 5240K 2-00:06:24 0:0 5240K 128 1 18-09:27:28 127:0 128 1 00:02:56 1936668K 18-09:27:28 127:0 2064284K 128 1 6-00:21:20 1:0 128 1 00:02:07 1869768K 6-00:21:20 1:0 2023584K 128 1 2-01:14:40 0:0 128 1 00:46:56 1:0 256 2 00:00:00 1080K 00:23:28 1:0 1080K 128 1 2-08:36:16 0:0 256 2 cd..

Colin Echeverría Aitken

Assistant Professor Biology Department Biochemistry Program Vassar College @. @.> 845.437.7430

On Nov 16, 2023, at 5:39 PM, Colin Aitken @.***> wrote:

Perhaps this is another way of testing it: I’ve sent the same job as previously (no reference transcriptome but with DE analysis) and assigned it to two nodes, giving it access to 256 CPU and 1024 GB memory.

Do you know of the best way to check on its memory usage? When I use squeue or sacct, I don’t get that info.

Colin Echeverría Aitken

Assistant Professor Biology Department Biochemistry Program Vassar College @. @.> 845.437.7430

On Nov 16, 2023, at 5:31 PM, Colin Aitken @.***> wrote:

Interesting … the question I have about that approach is that it already works without the DE analysis. And I need to input more samples to get the DE analysis to work, right? I suppose the other possibility would be to take a subset (just some of the fastq files with reads) of the four samples (2 conditions, 2 replicates).

Colin Echeverría Aitken

Assistant Professor Biology Department Biochemistry Program Vassar College @. @.> 845.437.7430

On Nov 16, 2023, at 2:51 PM, Sarah Griffiths @.***> wrote:

From the reports your data looks great, How large is the .mmi file in the failed minimap/samtools process.

I managed to create this error by not providing enough memory to this process. And it results in an empty bam. I wonder if when you had the memory directive set to 256GB and it was running 4 processes in parallel it was causing out of memory. You could also try just inputting one sample at a time to see if it still breaks - just to confirm its not a memory issue.

Caused by: Process pipeline:differential_expression:map_transcriptome (6) terminated with an error exit status (1)

Command executed:

minimap2 -t 4 -ax splice -uf -p 1.0 "genome_index.mmi" "seqs.fastq.gz" | samtools view -Sb > "output.bam" samtools sort -@ 4 "output.bam" -o "sample04_reads_aln_sorted.bam"

Command exit status: 1

Command output: (empty)

Command error: [main_samview] fail to read the header from "-". — Reply to this email directly, view it on GitHub https://github.com/epi2me-labs/wf-transcriptomes/issues/45#issuecomment-1815213081, or unsubscribe https://github.com/notifications/unsubscribe-auth/BD2SWUZL2UBWT2MY52PIIGTYEZVCZAVCNFSM6AAAAAA7M2VUXKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJVGIYTGMBYGE. You are receiving this because you were mentioned.

Just ran with 2 nodes: 256 CPUs and 1024 GB RAM. Same error. See below for .out and for the results of sacct on my most recent jobs. I’m not sure what to make of the mem=0 for the allocated resources.

3962 [d0/467650] process > output (15) [100%] 16 of 16 3963 ERROR ~ Error executing process > 'pipeline:differential_expression:map_transcriptome (4)' 3964 3965 Caused by: 3966 Process pipeline:differential_expression:map_transcriptome (4) terminated with an error exit status (1) 3967 3968 Command executed: 3969 3970 minimap2 -t 4 -ax splice -uf -p 1.0 "genome_index.mmi" "seqs.fastq.gz" | samtools view -Sb > "output.bam" 3971 samtools sort -@ 4 "output.bam" -o "WTpoly2_reads_aln_sorted.bam" 3972 3973 Command exit status: 3974 1 3975 3976 Command output: 3977 (empty) 3978 3979 Command error: 3980 [WARNING] Indexing parameters (-k, -w or -H) overridden by parameters used in the prebuilt index. 3981 [main_samview] fail to read the header from "-". 3982 3983 Work dir: 3984 /work/epi2me/work/52/699f2d864fe2104c6e34cff991d5cf 3985 3986 Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named . command.sh

 User    CPUTime     MaxRSS                        AllocTRES

caitken 00:21:20 cpu=128,energy=1844674407370955+ 00:21:20 1084K cpu=128,mem=0,node=1 caitken 00:23:28 cpu=128,energy=1844674407370955+ 00:23:28 1076K cpu=128,mem=0,node=1 caitken 00:04:16 cpu=128,energy=1844674407370955+ 00:04:16 1076K cpu=128,mem=0,node=1 caitken 00:04:16 cpu=128,energy=1844674407370955+ 00:04:16 1076K cpu=128,mem=0,node=1 caitken 00:34:08 cpu=128,energy=1844674407370955+ 00:34:08 1076K cpu=128,mem=0,node=1 caitken 00:19:12 cpu=128,energy=1844674407370955+ 00:19:12 1080K cpu=128,mem=0,node=1 caitken 5-22:19:44 cpu=128,energy=1844674407370955+ 5-22:19:44 2705508K cpu=128,mem=0,node=1 caitken 5-23:51:28 cpu=128,energy=1844674407370955+ 5-23:51:28 2574452K cpu=128,mem=0,node=1 caitken 04:35:12 5608K cpu=128,node=1,billing=128 caitken 2-00:06:24 5240K cpu=128,node=1,billing=128 caitken 18-09:27:28 cpu=128,energy=1844674407370955+ 18-09:27:28 2064284K cpu=128,mem=0,node=1 caitken 6-00:21:20 cpu=128,energy=1844674407370955+ 6-00:21:20 2023584K cpu=128,mem=0,node=1 caitken 5-23:42:56 cpu=128,energy=1844674407370955+ 5-23:42:56 2127260K cpu=128,mem=0,node=1 caitken 00:46:56 cpu=256,energy=1844674407370955+ 00:23:28 1080K cpu=128,mem=0,node=1 caitken 11-07:30:08 cpu=256,energy=1844674407370955+ 5-15:45:04 3188076K cpu=128,mem=0,node=1

Colin Echeverría Aitken

Assistant Professor Biology Department Biochemistry Program Vassar College @. @.> 845.437.7430

On Nov 16, 2023, at 5:59 PM, Colin Aitken @.***> wrote:

Index (.mmi) file is approximately 30 MB, I believe

Here is the memory usage for the jobs I’ve run today:
AveCPU     AveRSS    CPUTime ExitCode     MaxRSS      NCPUS   NNodes 
                    00:21:20    127:0                   128        1 
00:00:00 1084K 00:21:20 127:0 1084K 128 1 00:23:28 1:0 128 1 00:00:00 1076K 00:23:28 1:0 1076K 128 1 00:04:16 127:0 128 1 00:00:00 1076K 00:04:16 127:0 1076K 128 1 00:04:16 1:0 128 1 00:00:00 1076K 00:04:16 1:0 1076K 128 1 00:34:08 1:0 128 1 00:00:00 1076K 00:34:08 1:0 1076K 128 1 00:19:12 1:0 128 1 00:00:00 1080K 00:19:12 1:0 1080K 128 1 5-22:19:44 1:0 128 1 00:02:15 2210644K 5-22:19:44 1:0 2705508K 128 1 5-23:51:28 1:0 128 1 00:02:12 2574452K 5-23:51:28 1:0 2574452K 128 1 00:00:00 5608K 04:35:12 0:0 5608K 128 1 00:00:00 5240K 2-00:06:24 0:0 5240K 128 1 18-09:27:28 127:0 128 1 00:02:56 1936668K 18-09:27:28 127:0 2064284K 128 1 6-00:21:20 1:0 128 1 00:02:07 1869768K 6-00:21:20 1:0 2023584K 128 1 2-01:14:40 0:0 128 1 00:46:56 1:0 256 2 00:00:00 1080K 00:23:28 1:0 1080K 128 1 2-08:36:16 0:0 256 2 cd..

Colin Echeverría Aitken

Assistant Professor Biology Department Biochemistry Program Vassar College @. @.> 845.437.7430

On Nov 16, 2023, at 5:39 PM, Colin Aitken @.***> wrote:

Perhaps this is another way of testing it: I’ve sent the same job as previously (no reference transcriptome but with DE analysis) and assigned it to two nodes, giving it access to 256 CPU and 1024 GB memory.

Do you know of the best way to check on its memory usage? When I use squeue or sacct, I don’t get that info.

Colin Echeverría Aitken

Assistant Professor Biology Department Biochemistry Program Vassar College @. @.> 845.437.7430

On Nov 16, 2023, at 5:31 PM, Colin Aitken @.***> wrote:

Interesting … the question I have about that approach is that it already works without the DE analysis. And I need to input more samples to get the DE analysis to work, right? I suppose the other possibility would be to take a subset (just some of the fastq files with reads) of the four samples (2 conditions, 2 replicates).

Colin Echeverría Aitken

Assistant Professor Biology Department Biochemistry Program Vassar College @. @.> 845.437.7430

On Nov 16, 2023, at 2:51 PM, Sarah Griffiths @.***> wrote:

From the reports your data looks great, How large is the .mmi file in the failed minimap/samtools process.

I managed to create this error by not providing enough memory to this process. And it results in an empty bam. I wonder if when you had the memory directive set to 256GB and it was running 4 processes in parallel it was causing out of memory. You could also try just inputting one sample at a time to see if it still breaks - just to confirm its not a memory issue.

Caused by: Process pipeline:differential_expression:map_transcriptome (6) terminated with an error exit status (1)

Command executed:

minimap2 -t 4 -ax splice -uf -p 1.0 "genome_index.mmi" "seqs.fastq.gz" | samtools view -Sb > "output.bam" samtools sort -@ 4 "output.bam" -o "sample04_reads_aln_sorted.bam"

Command exit status: 1

Command output: (empty)

Command error: [main_samview] fail to read the header from "-". — Reply to this email directly, view it on GitHub https://github.com/epi2me-labs/wf-transcriptomes/issues/45#issuecomment-1815213081, or unsubscribe https://github.com/notifications/unsubscribe-auth/BD2SWUZL2UBWT2MY52PIIGTYEZVCZAVCNFSM6AAAAAA7M2VUXKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJVGIYTGMBYGE. You are receiving this because you were mentioned.

You can run just one sample up until the DE_analysis step, so this step that seems to breaking the workflow and the salmon counts will still work, maybe just try that to see if it works. You can look in the output folder and the execution folder and look at the report.html and timeline.html - it should show resources used up until the failure.

Understood. Will do. Thanks for sticking with me!Sent from my iPhone; please excuse typos. On Nov 17, 2023, at 3:59 AM, Sarah Griffiths @.***> wrote: You can run just one sample up until the DE_analysis step, so this step that seems to breaking the workflow and the salmon counts will still work, maybe just try that to see if it works. Ypu can look in the output folder and the execution folder and look at the report.html and timeline.html - it should show resources used up until the failure.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

So, the workflow checks that the sample sheet matches the number of samples. That means it fails if you remove the other samples (by pointing to the directory for just one barcode as the input). I then made a second directory containing directories for all four barcodes, but populated only one of those. But that fails when it gets to those reads and tries to preprocess those. Trying now with a sample sheet containing only one sample leads to different failures.

But, here’s an alternative approach I tried: I created subsetted versions of each of the four samples. Each barcode directory normally contains ~3000+ fastq.gzip files. I created versions of 100 files apiece and then ran the analysis using those. That avoids all the errors I mentioned previously (the good news). The bad news is that it fails in the same way, just more quickly (see below). I have posted the execution reports below. I tried to upload a zipped version of the work directory, but it was too large. Let me know if it would be helpful to see that. Thank you!!!

1939 ERROR ~ Error executing process > 'pipeline:differential_expression:map_transcriptome (4)' 1940 1941 Caused by: 1942 Process pipeline:differential_expression:map_transcriptome (4) terminated with an error exit status (1) 1943 1944 Command executed: 1945 1946 minimap2 -t 4 -ax splice -uf -p 1.0 "genome_index.mmi" "seqs.fastq.gz" | samtools view -Sb > "output.bam" 1947 samtools sort -@ 4 "output.bam" -o "WTpoly2_reads_aln_sorted.bam" 1948 1949 Command exit status: 1950 1 1951 1952 Command output: 1953 (empty) 1954 1955 Command error: 1956 [WARNING] Indexing parameters (-k, -w or -H) overridden by parameters used in the prebuilt index. 1957 [main_samview] fail to read the header from "-".

execution.zip

OK, here’s an alternative approach I tried: I created subsetted versions of each of the four samples. Each barcode directory normally contains ~3000+ fastq.gzip files. I created versions of 100 files apiece and then ran the analysis using those. That avoids all the errors I mentioned previously (the good news). The bad news is that it fails in the same way, just more quickly (see below). I will post the reports from execution and the files in the work directory separately.

Colin Echeverría Aitken

Assistant Professor Biology Department Biochemistry Program Vassar College @. @.> 845.437.7430

On Nov 17, 2023, at 8:02 AM, Colin Aitken @.***> wrote:

So, the workflow checks that the sample sheet matches the number of samples. That means it fails if you remove the other samples (by pointing to the directory for just one barcode as the input). I then made a second directory containing directories for all four barcodes, but populate only one of those. Which means it fails when it gets to those reads and tries to preprocess those (see below). Trying now with a sample sheet containing only one sample.

Last login: Thu Nov 16 20:18:23 on ttys000

The default interactive shell is now zsh. To update your account to use zsh, please run chsh -s /bin/zsh. For more details, please visit https://support.apple.com/kb/HT208050. caitken:~ caitken$ ssh @. @.'s password:

Welcome to Ubuntu 20.04.6 LTS (GNU/Linux 5.4.0-153-generic x86_64)

/ / / / __ / // / \/ \/ \/ _ \/ _/ / _ / // / // / // / / /
// //_/ ./ ./\//
// /_/

System information as of Fri 17 Nov 2023 07:44:26 AM EST

System load: 0.08 Users logged in: 1 Usage of /: 17.4% of 1.79TB IPv4 address for docker0: 172.17.0.1 Memory usage: 17% IPv4 address for eth0: 192.168.0.5 Swap usage: 0% IPv4 address for eth1: 143.229.7.20 Processes: 586 Last login: Thu Nov 16 20:18:35 2023 from 143.229.146.36 @.:~[07:44]$ cd /work/epi2me/ @.:/work/epi2me[07:44]$ check-my-quota @.'s password: Disk quotas for user caitken (uid 2019): Filesystem space quota limit grace files quota limit grace /home 13800M 25600M 30720M 1870 0 100k
/work 1438G 3200G 3500G 45455 0 100k
@.:/work/epi2me[07:44]$ ls 25 nextflow 50 outdir Aitken_epi2me_20231110_poly_emc_noDE.out output Aitken_epi2me_20231110_total_emc2.out Poly_epi2me_noRNA_sbatch Aitken_epi2me_20231110_total_emcK10.out Poly_epi2me_noRNA_sbatch_2nodes Aitken_epi2me_20231110_total_emcK11.out Poly_epi2me_noRNA_sbatch_lowMEM Aitken_epi2me_20231110_total_emcK14.out Poly_epi2me_sbatch_emc_DEonly Aitken_epi2me_20231110_total_emc_noDE.out Poly_epi2me_sbatch_emc_noDE.sh Aitken_epi2me_20231110_total_emc.out Poly_epi2me_sbatch_new Aitken_epi2me_poly_emc_DEonly.out test_data.tar.gz Aitken_epi2me_poly_noRNA_2nodes.out testDE_sbatch Aitken_epi2me_poly_noRNA2.out Total_epi2me_noRNA_sbatch Aitken_epi2me_poly_noRNA_lowMEM.out Total_epi2me_sbatch_emc Aitken_epi2me_poly_noRNA.out Total_epi2me_sbatch_emc_DEonly Aitken_epi2me_poly_plusRNA.out Total_epi2me_sbatch_emc_K10 Aitken_epi2me_testDE.out Total_epi2me_sbatch_emc_K11 Aitken_epi2me_total_emc_DEonly.out Total_epi2me_sbatch_emc_K14.sh Aitken_epi2me_total_noRNA.out Total_epi2me_sbatch_emc_NCBI.sh chr20 Total_epi2me_sbatch_emc_noDE differential_expression work differential_expression.tar.gz workspace_dir ERR6053095_chr20.fastq @.:/work/epi2me[07:44]$ cd /work/caitken/data/De -bash: cd: /work/caitken/data/De: No such file or directory @.:/work/epi2me[07:45]$ cd /work/caitken/data/DegronNanoporeSequencing/ @.:/work/caitken/data/DegronNanoporeSequencing[07:45]$ ls Barcodes.csv outputPoly Poly_epi2me_sbatch_general epi2me outputPoly_DE sacCer3 my_config_2nodes.cfg outputPoly_noRNA Total my_config.cfg outputTotal Total_epi2me_sbatch_general my_config_lowMEM.cfg outputTotal_DE work output outputTotal_noRNA work2 output2 Poly @.:/work/caitken/data/DegronNanoporeSequencing[07:45]$ vi my_config.cfg @.:/work/caitken/data/DegronNanoporeSequencing[07:45]$ cd Poly/ @.:/work/caitken/data/DegronNanoporeSequencing/Poly[07:47]$ ls barcode82 barcode84 barcode86 barcode88 @.:/work/caitken/data/DegronNanoporeSequencing/Poly[07:47]$ cd /work/epi2me/ @.:/work/epi2me[07:50]$ sbatch Poly_epi2me_noRNA_sbatch_1barcode Submitted batch job 36126 @.***:/work/epi2me[07:50]$ squeue --user=caitken --start --iterate=n_seconds squeue: error: --iterate=n_seconds

@.***:/work/epi2me[07:52]$ squeue --user=caitken --start --iterate=10 Fri Nov 17 07:52:48 2023 JOBID PARTITION NAME USER ST START_TIME NODES SCHEDNODES NODELIST(REASON)

Fri Nov 17 07:52:58 2023 JOBID PARTITION NAME USER ST START_TIME NODES SCHEDNODES NODELIST(REASON)

^C @.:/work/epi2me[07:53]$ ls 25 Aitken_epi2me_poly_plusRNA.out Poly_epi2me_sbatch_emc_DEonly 50 Aitken_epi2me_testDE.out Poly_epi2me_sbatch_emc_noDE.sh Aitken_epi2me_20231110_poly_emc_noDE.out Aitken_epi2me_total_emc_DEonly.out Poly_epi2me_sbatch_new Aitken_epi2me_20231110_total_emc2.out Aitken_epi2me_total_noRNA.out test_data.tar.gz Aitken_epi2me_20231110_total_emcK10.out chr20 testDE_sbatch Aitken_epi2me_20231110_total_emcK11.out differential_expression Total_epi2me_noRNA_sbatch Aitken_epi2me_20231110_total_emcK14.out differential_expression.tar.gz Total_epi2me_sbatch_emc Aitken_epi2me_20231110_total_emc_noDE.out ERR6053095_chr20.fastq Total_epi2me_sbatch_emc_DEonly Aitken_epi2me_20231110_total_emc.out nextflow Total_epi2me_sbatch_emc_K10 Aitken_epi2me_poly_emc_DEonly.out outdir Total_epi2me_sbatch_emc_K11 Aitken_epi2me_poly_noRNA_1barcode.out output Total_epi2me_sbatch_emc_K14.sh Aitken_epi2me_poly_noRNA_2nodes.out Poly_epi2me_noRNA_sbatch Total_epi2me_sbatch_emc_NCBI.sh Aitken_epi2me_poly_noRNA2.out Poly_epi2me_noRNA_sbatch_1barcode Total_epi2me_sbatch_emc_noDE Aitken_epi2me_poly_noRNA_lowMEM.out Poly_epi2me_noRNA_sbatch_2nodes work Aitken_epi2me_poly_noRNA.out Poly_epi2me_noRNA_sbatch_lowMEM workspace_dir @.:/work/epi2me[07:53]$ vi Aitken_epi2me_poly_noRNA_1barcode.out @.:/work/epi2me[07:53]$ cd /work/caitken/data/DegronNanoporeSequencing/ @.:/work/caitken/data/DegronNanoporeSequencing[07:54]$ ls Barcodes.csv output outputPoly_noRNA Poly_epi2me_sbatch_general work2 epi2me output2 outputTotal sacCer3 my_config_2nodes.cfg outputPoly outputTotal_DE Total my_config.cfg outputPoly_1barcode outputTotal_noRNA Total_epi2me_sbatch_general my_config_lowMEM.cfg outputPoly_DE Poly work @.:/work/caitken/data/DegronNanoporeSequencing[07:54]$ mkdir Poly1 @.:/work/caitken/data/DegronNanoporeSequencing[07:54]$ cp -r Poly/barcode82 Poly1/ @.:/work/caitken/data/DegronNanoporeSequencing[07:55]$ cd Poly1 @.:/work/caitken/data/DegronNanoporeSequencing/Poly1[07:56]$ ls barcode82 @.:/work/caitken/data/DegronNanoporeSequencing/Poly1[07:56]$ mkdir barcode84 @.:/work/caitken/data/DegronNanoporeSequencing/Poly1[07:56]$ mkdir barcode86 @.:/work/caitken/data/DegronNanoporeSequencing/Poly1[07:56]$ mkdir barcode88 @.:/work/caitken/data/DegronNanoporeSequencing/Poly1[07:56]$ ls barcode82 barcode84 barcode86 barcode88 @.:/work/caitken/data/DegronNanoporeSequencing/Poly1[07:56]$ cd barcode82 @.:/work/caitken/data/DegronNanoporeSequencing/Poly1/barcode82[07:56]$ ls PAO05278_pass_barcode82_9dbc3989_215283ee_0.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_324.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_100.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_325.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_101.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_326.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_102.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_327.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_103.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_328.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_104.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_329.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_105.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_32.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_106.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_330.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_107.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_331.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_108.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_332.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_109.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_333.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_10.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_334.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_110.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_335.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_111.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_336.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_112.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_337.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_113.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_338.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_114.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_339.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_115.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_33.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_116.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_340.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_117.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_341.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_118.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_342.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_119.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_343.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_11.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_344.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_120.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_345.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_121.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_346.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_122.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_347.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_123.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_348.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_124.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_349.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_125.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_34.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_126.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_350.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_127.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_351.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_128.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_352.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_129.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_353.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_12.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_354.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_130.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_355.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_131.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_356.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_132.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_357.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_133.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_358.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_134.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_359.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_135.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_35.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_136.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_360.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_137.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_361.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_138.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_362.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_139.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_363.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_13.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_364.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_140.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_365.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_141.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_366.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_142.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_367.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_143.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_368.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_144.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_369.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_145.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_36.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_146.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_370.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_147.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_371.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_148.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_372.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_149.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_373.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_14.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_374.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_150.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_375.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_151.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_376.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_152.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_377.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_153.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_378.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_154.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_379.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_155.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_37.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_156.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_380.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_157.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_381.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_158.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_382.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_159.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_383.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_15.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_384.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_160.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_385.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_161.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_386.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_162.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_387.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_163.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_388.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_164.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_389.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_165.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_38.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_166.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_390.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_167.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_391.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_168.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_392.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_169.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_393.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_16.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_394.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_170.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_395.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_171.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_396.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_172.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_397.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_173.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_398.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_174.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_399.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_175.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_39.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_176.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_3.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_177.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_400.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_178.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_401.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_179.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_402.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_17.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_403.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_180.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_404.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_181.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_405.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_182.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_406.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_183.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_407.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_184.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_408.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_185.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_409.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_186.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_40.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_187.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_410.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_188.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_411.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_189.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_412.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_18.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_413.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_190.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_414.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_191.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_415.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_192.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_416.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_193.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_417.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_194.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_418.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_195.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_419.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_196.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_41.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_197.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_420.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_198.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_421.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_199.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_422.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_19.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_423.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_1.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_424.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_200.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_425.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_201.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_426.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_202.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_427.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_203.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_428.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_204.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_429.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_205.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_42.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_206.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_430.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_207.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_431.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_208.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_432.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_209.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_433.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_20.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_434.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_210.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_435.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_211.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_436.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_212.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_437.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_213.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_438.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_214.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_439.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_215.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_43.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_216.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_440.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_217.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_441.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_218.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_442.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_219.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_443.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_21.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_444.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_220.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_445.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_221.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_446.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_222.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_447.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_223.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_448.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_224.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_449.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_225.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_44.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_226.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_450.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_227.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_451.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_228.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_452.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_229.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_453.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_22.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_454.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_230.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_455.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_231.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_456.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_232.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_457.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_233.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_458.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_234.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_459.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_235.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_45.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_236.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_460.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_237.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_461.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_238.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_462.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_239.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_463.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_23.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_464.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_240.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_465.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_241.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_466.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_242.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_467.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_243.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_468.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_244.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_469.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_245.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_46.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_246.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_470.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_247.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_471.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_248.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_472.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_249.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_473.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_24.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_474.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_250.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_475.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_251.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_476.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_252.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_477.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_253.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_478.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_254.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_479.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_255.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_47.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_256.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_480.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_257.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_481.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_258.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_482.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_259.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_483.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_25.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_484.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_260.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_485.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_261.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_486.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_262.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_487.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_263.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_488.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_264.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_489.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_265.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_48.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_266.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_490.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_267.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_491.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_268.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_492.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_269.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_493.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_26.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_494.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_270.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_495.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_271.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_496.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_272.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_497.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_273.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_49.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_274.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_4.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_275.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_50.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_276.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_51.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_277.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_52.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_278.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_53.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_279.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_54.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_27.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_55.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_280.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_56.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_281.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_57.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_282.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_58.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_283.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_59.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_284.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_5.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_285.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_60.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_286.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_61.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_287.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_62.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_288.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_63.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_289.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_64.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_28.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_65.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_290.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_66.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_291.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_67.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_292.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_68.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_293.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_69.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_294.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_6.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_295.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_70.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_296.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_71.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_297.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_72.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_298.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_73.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_299.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_74.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_29.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_75.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_2.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_76.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_300.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_77.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_301.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_78.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_302.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_79.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_303.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_7.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_304.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_80.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_305.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_81.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_306.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_82.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_307.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_83.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_308.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_84.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_309.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_85.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_30.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_86.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_310.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_87.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_311.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_88.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_312.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_89.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_313.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_8.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_314.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_90.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_315.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_91.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_316.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_92.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_317.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_93.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_318.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_94.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_319.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_95.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_31.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_96.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_320.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_97.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_321.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_98.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_322.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_99.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_323.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_9.fastq.gz @.:/work/caitken/data/DegronNanoporeSequencing/Poly1/barcode82[07:56]$ cd .. @.:/work/caitken/data/DegronNanoporeSequencing/Poly1[07:56]$ ls barcode82 barcode84 barcode86 barcode88 @.:/work/caitken/data/DegronNanoporeSequencing/Poly1[07:56]$ cd /work/epi2me/ @.:/work/epi2me[07:56]$ sbatch Poly_epi2me_noRNA_sbatch_1barcode Submitted batch job 36127 @.***:/work/epi2me[07:57]$ squeue --user=caitken --start --iterate=5 Fri Nov 17 07:57:34 2023 JOBID PARTITION NAME USER ST START_TIME NODES SCHEDNODES NODELIST(REASON)

Fri Nov 17 07:57:39 2023 JOBID PARTITION NAME USER ST START_TIME NODES SCHEDNODES NODELIST(REASON)

^C @.< 25 50 Aitken_epi2me_20231110_poly_emc_noDE.out Aitken_epi2me_20231110_total_emc2.out Aitken_epi2me_20231110_total_emcK10.out Aitken_epi2me_20231110_total_emcK11.out Aitken_epi2me_20231110_total_emcK14.out Aitken_epi2me_20231110_total_emc_noDE.out Aitken_epi2me_20231110_total_emc.out Aitken_epi2me_poly_emc_DEonly.out Aitken_epi2me_poly_noRNA_1barcode.out Aitken_epi2me_poly_noRNA_2nodes.out Aitken_epi2me_poly_noRNA2.out Aitken_epi2me_poly_noRNA_lowMEM.out Aitken_epi2me_poly_noRNA.out @.@.< JOBID PARTITION NAME USER ST 36127 emc Aitken_epi2me_po caitken R @.:/work/epi2me[07:57]$ ls Aitken_epi2me_poly_plusRNA.out Poly_epi2me_sbatch_emc_DEonly Aitken_epi2me_testDE.out Poly_epi2me_sbatch_emc_noDE.sh Aitken_epi2me_total_emc_DEonly.out Poly_epi2me_sbatch_new Aitken_epi2me_total_noRNA.out test_data.tar.gz chr20 testDE_sbatch differential_expression Total_epi2me_noRNA_sbatch differential_expression.tar.gz Total_epi2me_sbatch_emc ERR6053095_chr20.fastq Total_epi2me_sbatch_emc_DEonly nextflow Total_epi2me_sbatch_emc_K10 outdir Total_epi2me_sbatch_emc_K11 output Total_epi2me_sbatch_emc_K14.sh Poly_epi2me_noRNA_sbatch Total_epi2me_sbatch_emc_NCBI.sh Poly_epi2me_noRNA_sbatch_1barcode Total_epi2me_sbatch_emc_noDE Poly_epi2me_noRNA_sbatch_2nodes work Poly_epi2me_noRNA_sbatch_lowMEM workspace_dir rong>:/work/epi2me[07:57]$ vi Aitken_epi2me_poly_noRNA_1barcode.out em>:/work/epi2me[07:58]$ squeue TIME NODES CPUS NODELIST(REASON) 1:06 1 128 node5 rong>:/work/epi2me[07:58]$ squeue --user=caitken --start --iterate=5 START_TIME NODES SCHEDNODES NODELIST(REASON)

Fri Nov 17 07:58:34 2023 JOBID PARTITION NAME USER ST START_TIME NODES SCHEDNODES NODELIST(REASON)

:Fri Nov 17 07:58:39 2023 JOBID PARTITION NAME USER ST START_TIME NODES SCHEDNODES NODELIST(REASON)

^C @.:/work/epi2me[07:58]$ squeue --user=caitken JOBID PARTITION NAME USER ST TIME NODES CPUS NODELIST(REASON) @.:/work/epi2me[07:58]$ ^C @.***:/work/epi2me[07:59]$ squeue --user=general squeue: error: Invalid user: general

JOBID PARTITION NAME USER ST TIME NODES CPUS NODELIST(REASON)

@.:/work/epi2me[07:59]$ squeue JOBID PARTITION NAME USER ST TIME NODES CPUS NODELIST(REASON) @.:/work/epi2me[07:59]$ vi Aitken_epi2me_poly_noRNA_1barcode.out

32 sample_sheet : /work/caitken/BarcodesPoly.csv 33 34 Options for reference-based workflow 35 minimap2_index_opts: -w 50 36 37 Differential Expression Options 38 de_analysis : true 39 40 !! Only displaying parameters that differ from the pipeline defaults !! 41 -------------------------------------------------------------------------------- 42 If you use epi2me-labs/wf-transcriptomes for your analysis please cite: 43 44 * The nf-core framework 45 https://doi.org/10.1038/s41587-020-0439-x 46 47 48 -------------------------------------------------------------------------------- 49 This is epi2me-labs/wf-transcriptomes v0.4.1-g0a8fe19. 50 -------------------------------------------------------------------------------- 51 Checking fastq input. 52 [- ] process > validate_sample_sheet - 53 54 [- ] process > validate_sample_sheet - 55 [- ] process > fastcat - 56 [- ] process > pipeline:preprocess_ref_ann... - 57 [- ] process > pipeline:collectFastqIngres... - 58 [- ] process > pipeline:getVersions - 59 [- ] process > pipeline:getParams - 60 [- ] process > pipeline:preprocess_reads - 61 [- ] process > pipeline:build_minimap_index - 62 [- ] process > pipeline:reference_assembly... - 63 [- ] process > pipeline:split_bam - 64 [- ] process > pipeline:assemble_transcripts - 65 [- ] process > pipeline:merge_gff_bundles - 66 Doing reference based transcript analysis 67 68 [- ] process > validate_sample_sheet - 69 [- ] process > fastcat - 70 [- ] process > pipeline:preprocess_ref_ann... - 71 [- ] process > pipeline:collectFastqIngres... - 72 [- ] process > pipeline:getVersions - 73 [- ] process > pipeline:getParams - 74 [- ] process > pipeline:preprocess_reads - 75 [- ] process > pipeline:build_minimap_index - 76 [- ] process > pipeline:reference_assembly... - 76,1 8%

Colin Echeverría Aitken

Assistant Professor Biology Department Biochemistry Program Vassar College @. @.> 845.437.7430

On Nov 17, 2023, at 7:29 AM, Colin Aitken @.***> wrote:

Understood. Will do. Thanks for sticking with me!

Sent from my iPhone; please excuse typos.

On Nov 17, 2023, at 3:59 AM, Sarah Griffiths @.***> wrote:

You can run just one sample up until the DE_analysis step, so this step that seems to breaking the workflow and the salmon counts will still work, maybe just try that to see if it works. Ypu can look in the output folder and the execution folder and look at the report.html and timeline.html - it should show resources used up until the failure.

— Reply to this email directly, view it on GitHub https://github.com/epi2me-labs/wf-transcriptomes/issues/45#issuecomment-1815979427, or unsubscribe https://github.com/notifications/unsubscribe-auth/BD2SWU3LQ5MCHHVHOSVXVHTYE4RP5AVCNFSM6AAAAAA7M2VUXKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJVHE3TSNBSG4. You are receiving this because you were mentioned.

So, the workflow checks that the sample sheet matches the number of samples. That means it fails if you remove the other samples (by pointing to the directory for just one barcode as the input). I then made a second directory containing directories for all four barcodes, but populate only one of those. Which means it fails when it gets to those reads and tries to preprocess those (see below). Trying now with a sample sheet containing only one sample.

Last login: Thu Nov 16 20:18:23 on ttys000

Welcome to Ubuntu 20.04.6 LTS (GNU/Linux 5.4.0-153-generic x86_64)

/ / / / __ / // / \/ \/ \/ _ \/ _/ / _ / // / // / // / / /
// //_/ ./ ./\//
// /_/

System information as of Fri 17 Nov 2023 07:44:26 AM EST

System load: 0.08 Users logged in: 1 Usage of /: 17.4% of 1.79TB IPv4 address for docker0: 172.17.0.1 Memory usage: 17% IPv4 address for eth0: 192.168.0.5 Swap usage: 0% IPv4 address for eth1: 143.229.7.20 Processes: 586 Last login: Thu Nov 16 20:18:35 2023 from 143.229.146.36 @.:~[07:44]$ cd /work/epi2me/ @.:/work/epi2me[07:44]$ check-my-quota @.'s password: Disk quotas for user caitken (uid 2019): Filesystem space quota limit grace files quota limit grace /home 13800M 25600M 30720M 1870 0 100k
/work 1438G 3200G 3500G 45455 0 100k
@.:/work/epi2me[07:44]$ ls 25 nextflow 50 outdir Aitken_epi2me_20231110_poly_emc_noDE.out output Aitken_epi2me_20231110_total_emc2.out Poly_epi2me_noRNA_sbatch Aitken_epi2me_20231110_total_emcK10.out Poly_epi2me_noRNA_sbatch_2nodes Aitken_epi2me_20231110_total_emcK11.out Poly_epi2me_noRNA_sbatch_lowMEM Aitken_epi2me_20231110_total_emcK14.out Poly_epi2me_sbatch_emc_DEonly Aitken_epi2me_20231110_total_emc_noDE.out Poly_epi2me_sbatch_emc_noDE.sh Aitken_epi2me_20231110_total_emc.out Poly_epi2me_sbatch_new Aitken_epi2me_poly_emc_DEonly.out test_data.tar.gz Aitken_epi2me_poly_noRNA_2nodes.out testDE_sbatch Aitken_epi2me_poly_noRNA2.out Total_epi2me_noRNA_sbatch Aitken_epi2me_poly_noRNA_lowMEM.out Total_epi2me_sbatch_emc Aitken_epi2me_poly_noRNA.out Total_epi2me_sbatch_emc_DEonly Aitken_epi2me_poly_plusRNA.out Total_epi2me_sbatch_emc_K10 Aitken_epi2me_testDE.out Total_epi2me_sbatch_emc_K11 Aitken_epi2me_total_emc_DEonly.out Total_epi2me_sbatch_emc_K14.sh Aitken_epi2me_total_noRNA.out Total_epi2me_sbatch_emc_NCBI.sh chr20 Total_epi2me_sbatch_emc_noDE differential_expression work differential_expression.tar.gz workspace_dir ERR6053095_chr20.fastq @.:/work/epi2me[07:44]$ cd /work/caitken/data/De -bash: cd: /work/caitken/data/De: No such file or directory @.:/work/epi2me[07:45]$ cd /work/caitken/data/DegronNanoporeSequencing/ @.:/work/caitken/data/DegronNanoporeSequencing[07:45]$ ls Barcodes.csv outputPoly Poly_epi2me_sbatch_general epi2me outputPoly_DE sacCer3 my_config_2nodes.cfg outputPoly_noRNA Total my_config.cfg outputTotal Total_epi2me_sbatch_general my_config_lowMEM.cfg outputTotal_DE work output outputTotal_noRNA work2 output2 Poly @.:/work/caitken/data/DegronNanoporeSequencing[07:45]$ vi my_config.cfg @.:/work/caitken/data/DegronNanoporeSequencing[07:45]$ cd Poly/ @.:/work/caitken/data/DegronNanoporeSequencing/Poly[07:47]$ ls barcode82 barcode84 barcode86 barcode88 @.:/work/caitken/data/DegronNanoporeSequencing/Poly[07:47]$ cd /work/epi2me/ @.:/work/epi2me[07:50]$ sbatch Poly_epi2me_noRNA_sbatch_1barcode Submitted batch job 36126 @.***:/work/epi2me[07:50]$ squeue --user=caitken --start --iterate=n_seconds squeue: error: --iterate=n_seconds

@.***:/work/epi2me[07:52]$ squeue --user=caitken --start --iterate=10 Fri Nov 17 07:52:48 2023 JOBID PARTITION NAME USER ST START_TIME NODES SCHEDNODES NODELIST(REASON)

Fri Nov 17 07:52:58 2023 JOBID PARTITION NAME USER ST START_TIME NODES SCHEDNODES NODELIST(REASON)

^C @.:/work/epi2me[07:53]$ ls 25 Aitken_epi2me_poly_plusRNA.out Poly_epi2me_sbatch_emc_DEonly 50 Aitken_epi2me_testDE.out Poly_epi2me_sbatch_emc_noDE.sh Aitken_epi2me_20231110_poly_emc_noDE.out Aitken_epi2me_total_emc_DEonly.out Poly_epi2me_sbatch_new Aitken_epi2me_20231110_total_emc2.out Aitken_epi2me_total_noRNA.out test_data.tar.gz Aitken_epi2me_20231110_total_emcK10.out chr20 testDE_sbatch Aitken_epi2me_20231110_total_emcK11.out differential_expression Total_epi2me_noRNA_sbatch Aitken_epi2me_20231110_total_emcK14.out differential_expression.tar.gz Total_epi2me_sbatch_emc Aitken_epi2me_20231110_total_emc_noDE.out ERR6053095_chr20.fastq Total_epi2me_sbatch_emc_DEonly Aitken_epi2me_20231110_total_emc.out nextflow Total_epi2me_sbatch_emc_K10 Aitken_epi2me_poly_emc_DEonly.out outdir Total_epi2me_sbatch_emc_K11 Aitken_epi2me_poly_noRNA_1barcode.out output Total_epi2me_sbatch_emc_K14.sh Aitken_epi2me_poly_noRNA_2nodes.out Poly_epi2me_noRNA_sbatch Total_epi2me_sbatch_emc_NCBI.sh Aitken_epi2me_poly_noRNA2.out Poly_epi2me_noRNA_sbatch_1barcode Total_epi2me_sbatch_emc_noDE Aitken_epi2me_poly_noRNA_lowMEM.out Poly_epi2me_noRNA_sbatch_2nodes work Aitken_epi2me_poly_noRNA.out Poly_epi2me_noRNA_sbatch_lowMEM workspace_dir @.:/work/epi2me[07:53]$ vi Aitken_epi2me_poly_noRNA_1barcode.out @.:/work/epi2me[07:53]$ cd /work/caitken/data/DegronNanoporeSequencing/ @.:/work/caitken/data/DegronNanoporeSequencing[07:54]$ ls Barcodes.csv output outputPoly_noRNA Poly_epi2me_sbatch_general work2 epi2me output2 outputTotal sacCer3 my_config_2nodes.cfg outputPoly outputTotal_DE Total my_config.cfg outputPoly_1barcode outputTotal_noRNA Total_epi2me_sbatch_general my_config_lowMEM.cfg outputPoly_DE Poly work @.:/work/caitken/data/DegronNanoporeSequencing[07:54]$ mkdir Poly1 @.:/work/caitken/data/DegronNanoporeSequencing[07:54]$ cp -r Poly/barcode82 Poly1/ @.:/work/caitken/data/DegronNanoporeSequencing[07:55]$ cd Poly1 @.:/work/caitken/data/DegronNanoporeSequencing/Poly1[07:56]$ ls barcode82 @.:/work/caitken/data/DegronNanoporeSequencing/Poly1[07:56]$ mkdir barcode84 @.:/work/caitken/data/DegronNanoporeSequencing/Poly1[07:56]$ mkdir barcode86 @.:/work/caitken/data/DegronNanoporeSequencing/Poly1[07:56]$ mkdir barcode88 @.:/work/caitken/data/DegronNanoporeSequencing/Poly1[07:56]$ ls barcode82 barcode84 barcode86 barcode88 @.:/work/caitken/data/DegronNanoporeSequencing/Poly1[07:56]$ cd barcode82 @.:/work/caitken/data/DegronNanoporeSequencing/Poly1/barcode82[07:56]$ ls PAO05278_pass_barcode82_9dbc3989_215283ee_0.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_324.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_100.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_325.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_101.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_326.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_102.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_327.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_103.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_328.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_104.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_329.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_105.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_32.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_106.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_330.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_107.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_331.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_108.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_332.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_109.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_333.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_10.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_334.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_110.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_335.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_111.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_336.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_112.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_337.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_113.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_338.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_114.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_339.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_115.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_33.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_116.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_340.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_117.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_341.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_118.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_342.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_119.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_343.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_11.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_344.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_120.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_345.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_121.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_346.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_122.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_347.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_123.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_348.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_124.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_349.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_125.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_34.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_126.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_350.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_127.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_351.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_128.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_352.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_129.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_353.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_12.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_354.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_130.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_355.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_131.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_356.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_132.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_357.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_133.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_358.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_134.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_359.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_135.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_35.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_136.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_360.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_137.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_361.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_138.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_362.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_139.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_363.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_13.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_364.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_140.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_365.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_141.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_366.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_142.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_367.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_143.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_368.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_144.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_369.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_145.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_36.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_146.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_370.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_147.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_371.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_148.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_372.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_149.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_373.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_14.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_374.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_150.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_375.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_151.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_376.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_152.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_377.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_153.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_378.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_154.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_379.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_155.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_37.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_156.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_380.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_157.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_381.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_158.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_382.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_159.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_383.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_15.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_384.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_160.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_385.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_161.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_386.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_162.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_387.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_163.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_388.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_164.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_389.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_165.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_38.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_166.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_390.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_167.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_391.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_168.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_392.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_169.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_393.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_16.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_394.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_170.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_395.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_171.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_396.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_172.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_397.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_173.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_398.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_174.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_399.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_175.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_39.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_176.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_3.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_177.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_400.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_178.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_401.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_179.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_402.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_17.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_403.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_180.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_404.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_181.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_405.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_182.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_406.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_183.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_407.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_184.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_408.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_185.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_409.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_186.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_40.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_187.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_410.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_188.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_411.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_189.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_412.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_18.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_413.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_190.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_414.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_191.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_415.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_192.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_416.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_193.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_417.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_194.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_418.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_195.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_419.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_196.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_41.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_197.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_420.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_198.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_421.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_199.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_422.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_19.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_423.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_1.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_424.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_200.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_425.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_201.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_426.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_202.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_427.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_203.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_428.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_204.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_429.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_205.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_42.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_206.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_430.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_207.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_431.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_208.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_432.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_209.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_433.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_20.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_434.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_210.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_435.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_211.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_436.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_212.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_437.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_213.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_438.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_214.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_439.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_215.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_43.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_216.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_440.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_217.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_441.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_218.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_442.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_219.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_443.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_21.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_444.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_220.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_445.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_221.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_446.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_222.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_447.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_223.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_448.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_224.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_449.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_225.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_44.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_226.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_450.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_227.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_451.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_228.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_452.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_229.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_453.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_22.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_454.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_230.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_455.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_231.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_456.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_232.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_457.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_233.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_458.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_234.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_459.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_235.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_45.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_236.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_460.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_237.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_461.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_238.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_462.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_239.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_463.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_23.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_464.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_240.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_465.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_241.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_466.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_242.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_467.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_243.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_468.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_244.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_469.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_245.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_46.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_246.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_470.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_247.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_471.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_248.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_472.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_249.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_473.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_24.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_474.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_250.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_475.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_251.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_476.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_252.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_477.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_253.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_478.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_254.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_479.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_255.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_47.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_256.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_480.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_257.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_481.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_258.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_482.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_259.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_483.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_25.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_484.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_260.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_485.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_261.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_486.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_262.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_487.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_263.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_488.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_264.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_489.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_265.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_48.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_266.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_490.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_267.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_491.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_268.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_492.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_269.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_493.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_26.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_494.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_270.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_495.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_271.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_496.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_272.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_497.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_273.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_49.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_274.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_4.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_275.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_50.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_276.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_51.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_277.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_52.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_278.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_53.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_279.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_54.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_27.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_55.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_280.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_56.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_281.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_57.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_282.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_58.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_283.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_59.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_284.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_5.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_285.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_60.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_286.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_61.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_287.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_62.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_288.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_63.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_289.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_64.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_28.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_65.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_290.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_66.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_291.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_67.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_292.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_68.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_293.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_69.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_294.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_6.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_295.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_70.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_296.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_71.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_297.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_72.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_298.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_73.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_299.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_74.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_29.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_75.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_2.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_76.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_300.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_77.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_301.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_78.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_302.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_79.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_303.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_7.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_304.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_80.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_305.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_81.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_306.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_82.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_307.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_83.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_308.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_84.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_309.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_85.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_30.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_86.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_310.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_87.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_311.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_88.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_312.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_89.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_313.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_8.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_314.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_90.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_315.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_91.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_316.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_92.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_317.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_93.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_318.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_94.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_319.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_95.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_31.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_96.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_320.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_97.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_321.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_98.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_322.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_99.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_323.fastq.gz PAO05278_pass_barcode82_9dbc3989_215283ee_9.fastq.gz @.:/work/caitken/data/DegronNanoporeSequencing/Poly1/barcode82[07:56]$ cd .. @.:/work/caitken/data/DegronNanoporeSequencing/Poly1[07:56]$ ls barcode82 barcode84 barcode86 barcode88 @.:/work/caitken/data/DegronNanoporeSequencing/Poly1[07:56]$ cd /work/epi2me/ @.:/work/epi2me[07:56]$ sbatch Poly_epi2me_noRNA_sbatch_1barcode Submitted batch job 36127 @.***:/work/epi2me[07:57]$ squeue --user=caitken --start --iterate=5 Fri Nov 17 07:57:34 2023 JOBID PARTITION NAME USER ST START_TIME NODES SCHEDNODES NODELIST(REASON)

Fri Nov 17 07:57:39 2023 JOBID PARTITION NAME USER ST START_TIME NODES SCHEDNODES NODELIST(REASON)

^C @.:/work/epi2me[07:57]$ ls 25 Aitken_epi2me_poly_plusRNA.out Poly_epi2me_sbatch_emc_DEonly 50 Aitken_epi2me_testDE.out Poly_epi2me_sbatch_emc_noDE.sh Aitken_epi2me_20231110_poly_emc_noDE.out Aitken_epi2me_total_emc_DEonly.out Poly_epi2me_sbatch_new Aitken_epi2me_20231110_total_emc2.out Aitken_epi2me_total_noRNA.out test_data.tar.gz Aitken_epi2me_20231110_total_emcK10.out chr20 testDE_sbatch Aitken_epi2me_20231110_total_emcK11.out differential_expression Total_epi2me_noRNA_sbatch Aitken_epi2me_20231110_total_emcK14.out differential_expression.tar.gz Total_epi2me_sbatch_emc Aitken_epi2me_20231110_total_emc_noDE.out ERR6053095_chr20.fastq Total_epi2me_sbatch_emc_DEonly Aitken_epi2me_20231110_total_emc.out nextflow Total_epi2me_sbatch_emc_K10 Aitken_epi2me_poly_emc_DEonly.out outdir Total_epi2me_sbatch_emc_K11 Aitken_epi2me_poly_noRNA_1barcode.out output Total_epi2me_sbatch_emc_K14.sh Aitken_epi2me_poly_noRNA_2nodes.out Poly_epi2me_noRNA_sbatch Total_epi2me_sbatch_emc_NCBI.sh Aitken_epi2me_poly_noRNA2.out Poly_epi2me_noRNA_sbatch_1barcode Total_epi2me_sbatch_emc_noDE Aitken_epi2me_poly_noRNA_lowMEM.out Poly_epi2me_noRNA_sbatch_2nodes work Aitken_epi2me_poly_noRNA.out Poly_epi2me_noRNA_sbatch_lowMEM workspace_dir @.:/work/epi2me[07:57]$ vi Aitken_epi2me_poly_noRNA_1barcode.out @.:/work/epi2me[07:58]$ squeue JOBID PARTITION NAME USER ST TIME NODES CPUS NODELIST(REASON) 36127 emc Aitken_epi2me_po caitken R 1:06 1 128 node5 @.:/work/epi2me[07:58]$ squeue --user=caitken --start --iterate=5 Fri Nov 17 07:58:29 2023 JOBID PARTITION NAME USER ST START_TIME NODES SCHEDNODES NODELIST(REASON)

Fri Nov 17 07:58:34 2023 JOBID PARTITION NAME USER ST START_TIME NODES SCHEDNODES NODELIST(REASON)

:Fri Nov 17 07:58:39 2023 JOBID PARTITION NAME USER ST START_TIME NODES SCHEDNODES NODELIST(REASON)

^C @.:/work/epi2me[07:58]$ squeue --user=caitken JOBID PARTITION NAME USER ST TIME NODES CPUS NODELIST(REASON) @.:/work/epi2me[07:58]$ ^C @.***:/work/epi2me[07:59]$ squeue --user=general squeue: error: Invalid user: general

         JOBID PARTITION             NAME     USER ST       TIME  NODES   CPUS NODELIST(REASON)

@.:/work/epi2me[07:59]$ squeue JOBID PARTITION NAME USER ST TIME NODES CPUS NODELIST(REASON) @.:/work/epi2me[07:59]$ vi Aitken_epi2me_poly_noRNA_1barcode.out

32 sample_sheet : /work/caitken/BarcodesPoly.csv 33 34 Options for reference-based workflow 35 minimap2_index_opts: -w 50 36 37 Differential Expression Options 38 de_analysis : true 39 40 !! Only displaying parameters that differ from the pipeline defaults !! 41 -------------------------------------------------------------------------------- 42 If you use epi2me-labs/wf-transcriptomes for your analysis please cite: 43 44 * The nf-core framework 45 https://doi.org/10.1038/s41587-020-0439-x 46 47 48 -------------------------------------------------------------------------------- 49 This is epi2me-labs/wf-transcriptomes v0.4.1-g0a8fe19. 50 -------------------------------------------------------------------------------- 51 Checking fastq input. 52 [- ] process > validate_sample_sheet - 53 54 [- ] process > validate_sample_sheet - 55 [- ] process > fastcat - 56 [- ] process > pipeline:preprocess_ref_ann... - 57 [- ] process > pipeline:collectFastqIngres... - 58 [- ] process > pipeline:getVersions - 59 [- ] process > pipeline:getParams - 60 [- ] process > pipeline:preprocess_reads - 61 [- ] process > pipeline:build_minimap_index - 62 [- ] process > pipeline:reference_assembly... - 63 [- ] process > pipeline:split_bam - 64 [- ] process > pipeline:assemble_transcripts - 65 [- ] process > pipeline:merge_gff_bundles - 66 Doing reference based transcript analysis 67 68 [- ] process > validate_sample_sheet - 69 [- ] process > fastcat - 70 [- ] process > pipeline:preprocess_ref_ann... - 71 [- ] process > pipeline:collectFastqIngres... - 72 [- ] process > pipeline:getVersions - 73 [- ] process > pipeline:getParams - 74 [- ] process > pipeline:preprocess_reads - 75 [- ] process > pipeline:build_minimap_index - 76 [- ] process > pipeline:reference_assembly... - 76,1 8%

Colin Echeverría Aitken

Assistant Professor Biology Department Biochemistry Program Vassar College @. @.> 845.437.7430

On Nov 17, 2023, at 7:29 AM, Colin Aitken @.***> wrote:

Understood. Will do. Thanks for sticking with me!

Sent from my iPhone; please excuse typos.

On Nov 17, 2023, at 3:59 AM, Sarah Griffiths @.***> wrote:

You can run just one sample up until the DE_analysis step, so this step that seems to breaking the workflow and the salmon counts will still work, maybe just try that to see if it works. Ypu can look in the output folder and the execution folder and look at the report.html and timeline.html - it should show resources used up until the failure.

— Reply to this email directly, view it on GitHub https://github.com/epi2me-labs/wf-transcriptomes/issues/45#issuecomment-1815979427, or unsubscribe https://github.com/notifications/unsubscribe-auth/BD2SWU3LQ5MCHHVHOSVXVHTYE4RP5AVCNFSM6AAAAAA7M2VUXKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJVHE3TSNBSG4. You are receiving this because you were mentioned.

Just checking in to provide some updates and ideas:

1. I see these lines in the begin Ning of my nextflow logs:

89 Nov-17 14:56:41.057 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=4; memory=8 GB; capacity=128; pollInterval=100ms; dumpInterval=5m 90 Nov-17 14:56:41.060 [main] DEBUG n.processor.TaskPollingMonitor - >>> barrier register (monitor: local)

**This is despite passing a config file requesting 128 cpus and 512 GB RAM

This is what nextflow appears to see for the node I assigned it to:**

27 Version: 23.10.0 build 5889 28 Created: 15-10-2023 15:07 UTC (11:07 EDT) 29 System: Linux 5.4.0-153-generic 30 Runtime: Groovy 3.0.19 on OpenJDK 64-Bit Server VM 11.0.19+7-post-Ubuntu-0ubuntu120.04.1 31 Encoding: UTF-8 (UTF-8) 32 Process: 1102198@node3 [192.168.0.30] 33 CPUs: 128 - Mem: 503.6 GB (455.6 GB) - Swap: 32 GB (32 GB)

Which raises a question for me: is there any chance docker is not requesting the configuration I am specifying? The line in my sbatch script pointing to the config.cfg file is below:

-c /work/caitken/data/DegronNanoporeSequencing/my_config.cfg

and the contents of my_config.cfg are:

1 executor { 2 $local { 3 cpus = 128 4 memory = "512 GB" 5 } 6 }

2. The second observation/idea is that there might be memory limits that are not properly set on our HPC. If I request a specific memory allocation using srun or sbatch, it only works if I request 1 MB or less. I have been using #SBATCH --mem=MaxMemPerNode but when I look at my runs, they appear to be using very little physical memory and tons of virtual memory. See below for sbatch code and sinfo results

sbatch code

!/bin/bash

SBATCH -J Aitken_epi2me_poly_noRNA_subset

SBATCH -o Aitken_epi2me_poly_noRNA_subset.out

SBATCH --nodes=1

SBATCH --exclusive

SBATCH --mem=MaxMemPerNode

SBATCH --nodelist=node3

SBATCH --mail-user=caitken@vassar.edu

SBATCH --mail-type=begin

SBATCH --mail-type=end

cd /work/epi2me

./nextflow -log /work/caitken/cea_nextflow.log run epi2me-labs/wf-transcriptomes -with-trace \ --fastq /work/caitken/data/DegronNanoporeSequencing/Poly1/ \ --ref_genome /work/caitken/data/DegronNanoporeSequencing/sacCer3/20110902_sacCer3.fa \ --ref_annotation /work/caitken/data/DegronNanoporeSequencing/sacCer3/20110902_sacCer3.gff \ --pychopper_opts '-k PCS111' --minimap2_index_opts "-w 50" \ --de_analysis \ --sample_sheet /work/caitken/BarcodesPoly.csv \ --out_dir /work/caitken/data/DegronNanoporeSequencing/outputPoly_subset \ -c /work/caitken/data/DegronNanoporeSequencing/my_config.cfg

sinfo results

caitken@ccas020:/work/epi2me[02:49]$ sinfo --Node --format=="%.10N %.6D %.10P %.11T %.4c %.8z %.6m %.10e %.8d %.6w %.8f %20E" = NODELIST NODES PARTITION STATE CPUS S:C:T MEMORY FREE_MEM TMP_DISK WEIGHT AVAIL_FE REASON
= node1 1 general idle 48 8:6:1 1 4733 0 1 (null) none
= node2 1 general idle 48 8:6:1 1 7425 0 1 (null) none
= node3 1 general* idle 128 8:8:2 1 467062 0 1 (null) none
= node5 1 emc idle 128 8:8:2 1 5618 0 1 (null) none
= node6 1 emc idle 128 8:8:2 1 22051 0 1 (null) none
= node7 1 emc idle 128 8:8:2 1 317002 0 1 (null) none
= node8 1 emc idle 128 8:8:2 1 437312 0 1 (null) none

OK, I can rule out at least one aspect of this limited memory hypothesis. I was able to get our admin to change the memory limits on the nodes (see below). But when I run the job it is still using almost exclusively virtual memory, not physical memory (at least insofar as I can tell).

sinfo

caitken@ccas020:/work/epi2me[05:07]$ sinfo --Node --format=="%.10N %.6D %.10P %.11T %.4c %.8z %.6m %.10e %.8d %.6w %.8f %20E" = NODELIST NODES PARTITION STATE CPUS S:C:T MEMORY FREE_MEM TMP_DISK WEIGHT AVAIL_FE REASON
= node1 1 general idle 48 8:6:1 61440 4826 0 1 (null) none
= node2 1 general idle 48 8:6:1 61440 7478 0 1 (null) none
= node3 1 general* idle 128 8:8:2 512000 457538 0 1 (null) none
= node5 1 emc allocated 128 8:8:2 512000 1607 0 1 (null) none
= node6 1 emc allocated 128 8:8:2 512000 22231 0 1 (null) none
= node7 1 emc idle 128 8:8:2 512000 317105 0 1 (null) none
= node8 1 emc idle 128 8:8:2 512000 437388 0 1 (null) none

htop on running job

PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND

1270399 caitken 20 0 79580 4468 3012 S 88.4 0.0 0:02.68 nhmmscan
1270402 caitken 20 0 79644 4444 2960 S 88.1 0.0 0:02.67 nhmmscan
1270405 caitken 20 0 79560 4272 2808 S 87.8 0.0 0:02.66 nhmmscan
1270410 caitken 20 0 79416 4380 2952 S 87.5 0.0 0:02.65 nhmmscan
1271562 root 20 0 0 0 0 R 31.4 0.0 0:00.95 192.168.40.3-ma
1247115 root 20 0 0 0 0 I 16.5 0.0 0:00.50 kworker/44:2-events

Could you go to the work directory of the merge_transcriptomes and if its small enough perhaps zip up and send the final_non_redundant_transcriptome.fasta and if possible also the gtf file from that directory? Just to check it looks sensible?

Files attached below.

Some updates on where things stand here (updating my posts above). I've been chasing the memory constraint lead. I realized that the jobs I'm running on our HPC our running entirely on virtual memory and using almost no RSS. So for example, the results of htop on the running node show almost every process running at 48 GB of virtual memory and using only 0.4% of available memory (I can provide a screen grab of that if you want). In trying to chase this down, I realized that the memory limits within SLURM were not set properly. Unfortunately, correcting those (I can now ask SLRUM to give me all 512 GB of RAM on the node and it will) still result in the same odd virtual memory usage (unless this is actually normal?) and the processes still fail in the same way. I also noticed that it seemed like nextflow/docker were ignoring my config file that I was passing (I still saw what looked like 4 CPU and 8 GB of ram ... the default ... in the nextflow log). So I modified nextflow.log just to change those two lines to match what I'm requesting (128 CPU and 512 GB of RAM). But still the same failure. So that brings me to my question: Is this virtual memory usage, and the very low physical memory usage expected? If not, do you have ideas of where to look? I'd like to make as much progress as possible investigating solutions on our end, as I know you're very busy!

TXandGTF.zip

ok those files look reasonable. Its only a very small transcriptome so although you have a lot of reads, I wouldn't expect this step to require that much memory. Could you possibly try to run the workflow on a laptop with docker or singularity? You could even try the desktop app EPI2ME https://labs.epi2me.io/downloads/. Asking my team if they have any ideas about your SLURM issues.

I will try … I have tried previously to run in on a PC, but it takes absolutely forever. I was able to get it to run on one barcode, but it took 44 hours. I then tried with everything together, but I got errors I had not seen previously. I have also tried running it on my Mac (Mac Studio), as it’s relatively powerful. But the desktop app does not play well with the new apple silicon processors … would that be any different if I run through the terminal?

I will try running with this smaller dataset on our PC and let you know what I see.

Thanks for sticking with me!

Colin Echeverría Aitken

Assistant Professor Biology Department Biochemistry Program Vassar College @. @.> 845.437.7430

On Nov 20, 2023, at 9:25 AM, Sarah Griffiths @.***> wrote:

ok those files look reasonable. Its only a very small transcriptome so although you have a lot of reads, I wouldn't expect this step to require that much memory. Could you possibly try to run the workflow on a laptop with docker or singularity? You could even try the desktop app EPI2ME https://labs.epi2me.io/downloads/. Asking my team if they have any ideas about your SLURM issues.

— Reply to this email directly, view it on GitHub https://github.com/epi2me-labs/wf-transcriptomes/issues/45#issuecomment-1819165213, or unsubscribe https://github.com/notifications/unsubscribe-auth/BD2SWU5TXGVWN7JDEQRJ3XDYFNR6PAVCNFSM6AAAAAA7M2VUXKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJZGE3DKMRRGM. You are receiving this because you were mentioned.

ok ... I just did this with a further subsetted dataset (only 10 fastq.gz files per barcode). Ran on a local PC using desktop GUI ... received the exact same error (log below). I ran this further subsetted version on our HPC and again, same error. I then tried on the local PC again, but using different genome files (I had previously tested this on our HPC but that was before I realized I should not provide transcriptome when using DE and instead rely on generated transcriptome).

The good news: it looks like it proceeds past the point where we've seen the error previously. The bad news: it now fails at a separate step ... the error suggests it is looking for "+" and "-" in the gff/gtf granges object but doesn't find it (logs below ... labeled Genbank and RefSeq). Do you think we can move forward from here?

A few questions/ideas:

The desktop GUI says that a reference transcriptome is required for DE analysis ... but didn't complain that I did not provide one ... could this be an issue? My understanding was that if one chose reference-guided, the workflow assemblies a transcriptome that then guides the subsequent DE analysis (a question here ... is a separate transcriptome generated per sample? or is there a master merged transcriptome that is used for all samples for the DE analysis?)
I had to change one of the options in the GUI version ... to say that alignments needed to be in at least 2 samples minimum (default was three) ... because I only have two replicates of each ... could that be an issue?

nextflow.txt RefSeq_epi2me.txt GenBank_epi2me.txt

just check on our HPC and the same holds true there ... it runs further with genbank files but crashes at the pipeline:differential_expression:deAnalysis(1) step (see below)

2189 [b6/d0517e] process > output (16) [ 69%] 16 of 23 2190 Retry deAnalysis with gff format setting and version removal. 2191 Retry deAnalysis with gtf format setting and version removal. 2192 [70/24e7e0] NOTE: Process pipeline:differential_expression:deAnalysis (1) terminated with an error exit status (1) -- Execution is retried (2) 2193 [94/280d99] NOTE: Process pipeline:differential_expression:deAnalysis (1) terminated with an error exit status (1) -- Execution is retried (3) 2194 ERROR ~ Error executing process > 'pipeline:differential_expression:deAnalysis (1)' 2195 2196 Caused by: 2197 Process pipeline:differential_expression:deAnalysis (1) terminated with an error exit status (1) 2198 2199 Command executed: 2200 2201 mkdir merged 2202 mkdir de_analysis 2203 mv de_transcript_counts.tsv merged/all_counts.tsv 2204 mv BarcodesPoly.csv de_analysis/coldata.tsv 2205 de_analysis.R annotation.gtf 3 1 10 3 gtf true 2206 2207 Command exit status: 2208 1 2209 2210 Command output: 2211 Loading counts, conditions and parameters. 2212 Loading annotation database. 2213 2214 Command error: 2215 Loading counts, conditions and parameters. 2216 Warning message: 2217 In read.table(file = file, header = header, sep = sep, quote = quote, : 2218 incomplete final line found by readTableHeader on 'de_analysis/coldata.tsv' 2219 Loading annotation database. 2220 Import genomic features from the file as a GRanges object ... OK 2221 Prepare the 'metadata' data frame ... OK 2222 Make the TxDb object ... Error in .makeTxDb_normarg_transcripts(transcripts) : 2223 values in 'transcripts$tx_strand' must be "+" or "-" 2224 Calls: makeTxDbFromGFF ... makeTxDbFromGRanges -> makeTxDb -> .makeTxDb_normarg_transcripts 2225 Execution halted

Ah no I will update that about the reference transcriptome requirement. Oh that's good, we have only tested it as mentioned in the documentation with annotation files from Encode, Ensembl and NCBI.

I think just ensure the strand column in your ref annotation file only has + or - or i think . is ok as well. Let me know if you need any guidance on fixing it, feel free to send the ref_annotation you are using or a link to where its from and I can check it. No it should be ok to set that it needs to be present in 2 samples.

One thing to try may be to reduce the threads parameter to one or two, which will make some steps take a bit longer but may reduce memory requirement at the transcriptome alignment step.

Also in the output directory execution folder did you have any example report.html and timeline.htmls, do they have any info about memory consumption before the workflow failure?

Attaching the ones I tried that work ... one of these is NCBI_RefSeq so should work .... but I see "." for entries like centromeres and telomeres ... suggestions on how to fix (I don't remember my Perl for text editing all that well)? Or alternatives for yeast?

GenBank.zip NCBI_RefSeq.zip

Here is the execution folder. Bear in mind this is the heavily downsampled data set (only 10 files per barcode), it's likely not testing memory limits like the others. That said, it still is eating up 80 GB of virtual memory on the HPC ... and only 1 G RAM ... is that normal?

I will say that for the other jobs, it was only the preprocessing reads step that appeared hugely memory intensive, at least according to these reports. And that always worked fine on the HPC (whereas it takes ages on a local PC)

execution.zip

I think any "." entries are ok, have you tried the genomic.gtf as the annotation file from the genbank zip folder, Oh I also noticed ### at the end of each of the annotation files.. I have not seen this before. Perhaps remove the last lines.

epi2me-labs / wf-transcriptomes

samtools error ([main_samview] fail to read the header from "-") during DE analysis ... empty bam file? #45

Ask away!

!/bin/bash

SBATCH -J Aitken_epi2me_poly_noRNA

SBATCH -o Aitken_epi2me_poly_noRNA.out

SBATCH --nodes=1

SBATCH --exclusive

SBATCH --mem=MaxMemPerNode

SBATCH -p emc

|||||||||| _ __ _ ** |||||||||| | __| _ | | \/ | __| | | | | ||||| | | | |) | | ) | |\/| | _| ___| |/ ` | ' \/ | ||||| | |_| /| | / /| | | | |_|| | (| | |) _ \ |||||||||| |____|_| |_|___|| ||| ||\,|._/|/ |||||||||| wf-transcriptomes v0.4.1-g0a8fe19

!! Only displaying parameters that differ from the pipeline defaults !!

This is epi2me-labs/wf-transcriptomes v0.4.1-g0a8fe19.

!/bin/bash

SBATCH -J Aitken_epi2me_poly_noRNA_subset

SBATCH -o Aitken_epi2me_poly_noRNA_subset.out

SBATCH --nodes=1

SBATCH --exclusive

SBATCH --mem=MaxMemPerNode

SBATCH --nodelist=node3

SBATCH --mail-user=caitken@vassar.edu

SBATCH --mail-type=begin

SBATCH --mail-type=end

epi2me-labs / wf-transcriptomes

samtools error ([main_samview] fail to read the header from "-") during DE analysis ... empty bam file? #45

Ask away!

!/bin/bash

SBATCH -J Aitken_epi2me_poly_noRNA

SBATCH -o Aitken_epi2me_poly_noRNA.out

SBATCH --nodes=1

SBATCH --exclusive

SBATCH --mem=MaxMemPerNode

SBATCH -p emc

|||||||||| _ __ _ __ ____ |||||||||| | ____| _ | | \/ | __| | | | | ||||| | | | |) | | ) | |\/| | _| ___| |/ ` | ' \/ | ||||| | |_| /| | / /| | | | |_|| | (| | |) _ \ |||||||||| |____|_| |_|___|| ||| ||\,|._/|/ |||||||||| wf-transcriptomes v0.4.1-g0a8fe19

!! Only displaying parameters that differ from the pipeline defaults !!

This is epi2me-labs/wf-transcriptomes v0.4.1-g0a8fe19.

!/bin/bash

SBATCH -J Aitken_epi2me_poly_noRNA_subset

SBATCH -o Aitken_epi2me_poly_noRNA_subset.out

SBATCH --nodes=1

SBATCH --exclusive

SBATCH --mem=MaxMemPerNode

SBATCH --nodelist=node3

SBATCH --mail-user=caitken@vassar.edu

SBATCH --mail-type=begin

SBATCH --mail-type=end

|||||||||| _ __ _ ** |||||||||| | __| _ | | \/ | __| | | | | ||||| | | | |) | | ) | |\/| | _| ___| |/ ` | ' \/ | ||||| | |_| /| | / /| | | | |_|| | (| | |) _ \ |||||||||| |____|_| |_|___|| ||| ||\,|._/|/ |||||||||| wf-transcriptomes v0.4.1-g0a8fe19