search

epi2me-labs / wf-transcriptomes

Other
64 stars 30 forks source link

Buil_minimap_index_transcriptome error #64

Closed mdhitch closed 3 months ago

mdhitch commented 5 months ago

Ask away!

On several machines I am using I am getting an error at the build_minimap_transcriptome stage. Errors with a 137 exit status suggesting a memory issue. However, some of the machines I have tried with have quite some substantial resources and all still fail at this step. I don’t believe this step is using all resources that have been allocated, so would be grateful to have some advice on getting over this hurdle.

Just as a side issue, working with ~16 samples and ~ just over 190 million reads, the whole workflow to this stage takes quite some time too, so any advice on speeding things up would also be much appreciated.

Best

Matt

nrhorner commented 5 months ago

Hi @mdhitch

Which reference transcriptome are you using (--ref_transcriptome). Is it publicly available? If not, could you let me how many sequences are in it?

Could you post a log too please?

Thanks,

Neil

mdhitch commented 5 months ago

Hi @nrhorner,

This particular one is from the gird. N E X T F L O W ~ version 23.04.2 Launching/data/scratch/workflows/epi2me-labs/wf-transcriptomes/main.nf[brave_einstein] DSL2 - revision: 0b8606d864 |||||||||| _____ ____ ___ ____ __ __ _____ _ _ |||||||||| | ____| _ \_ _|___ \| \/ | ____| | | __ _| |__ ___ ||||| | _| | |_) | | __) | |\/| | _| _____| |/ _ | ' \/ | ||||| | || /| | / /| | | | ||_| | (| | |) \ \ |||||||||| |_|_| |_|___|| ||| ||\,|._/|/ |||||||||| wf-transcriptomes v1.0.0

Core Nextflow options runName : brave_einstein containerEngine: docker launchDir : /data/scratch/instances/wf-transcriptomes_01HNFWF1ETA50SKYX76XTK4CZQ workDir : /data/scratch/instances/wf-transcriptomes_01HNFWF1ETA50SKYX76XTK4CZQ/work projectDir : /data/scratch/workflows/epi2me-labs/wf-transcriptomes userName : grid profile : standard configFiles : /data/scratch/workflows/epi2me-labs/wf-transcriptomes/nextflow.config, /data/scratch/instances/wf-transcriptomes_01HNFWF1ETA50SKYX76XTK4CZQ/local.config Input Options fastq : /data/scratch/nano_trans/fastqs ref_genome : /data/Adaptive_Sampling_Refs/my_jaffa/hg38.fa ref_annotation : /data/Adaptive_Sampling_Refs/my_jaffa/genCode22.annotation.gtf Output Options out_dir : /data/scratch/instances/wf-transcriptomes_01HNFWF1ETA50SKYX76XTK4CZQ/output Sample Options sample_sheet : /data/scratch/jeff_conditions.csv Differential Expression Options de_analysis : true Advanced Options cdna_kit : SQK-PCS111 !! Only displaying parameters that differ from the pipeline defaults !!

If you use epi2me-labs/wf-transcriptomes for your analysis please cite:

Genome is the human gencode22 version (thought it vital to use this version at first to line up with jaffa but those previous issues mean its not).

Best

Matt

@nrhorner, just noticed that final_non_redundant_transcriptome.fasta is empty.

nrhorner commented 5 months ago

Hi @mdhitch

There is a fix on it's way for this.

In the meantime a potential workaround is to override the memory directive for the failing process. To do this, create a file containing this (lets call it mem.config) 

process {
    withName: build_minimap_index_transcriptome {
        memory = 32.GB
    }
}

then apply it to the workflow using -c mem.config

let me knwo if that does not work for you.

Thanks,

Neil

mdhitch commented 5 months ago

Thank you @nrhorner . I will give it a go. Blatantly obvious that I am (I am afraid to say) relatively new to using Nextflow and as such tuning parameters to suit.

sarahjeeeze commented 4 months ago

Hi, this should now be updated in v1.1.0 - let us know if you still have issues.

sarahjeeeze commented 3 months ago

Commenting through lack of response and issue should now be resolved.