Error: EXITING: Did not find the genome in memory, did not remove any genomes from shared memory

[lenshinaagbor]

Hi, I just came across this software as I have been struggling with MAKER. Running the example data and my data, I have encountered this same isssue. It says it can't find the genome but it's right there.

EXITING: Did not find the genome in memory, did not remove any genomes from shared memory This is what is on the log.out file;

cat Log.out STAR version=2.7.7a STAR compilation time,server,dir=Mon Dec 28 13:38:40 EST 2020 vega:/home/dobin/data/STAR/STARcode/STAR.master/source

Command Line: STAR --runThreadN 30 --genomeLoad Remove --genomeDir /mnt/nfs/home/b9017460/finder/example/star_index_without_transcriptome

Initial USER parameters from Command Line: All USER parameters from Command Line: runThreadN 30 ~RE-DEFINED genomeLoad Remove ~RE-DEFINED genomeDir /mnt/nfs/home/b9017460/finder/example/star_index_without_transcriptome ~RE-DEFINED

Finished reading parameters from all sources Final user re-defined parameters-----------------: runThreadN 30 genomeDir /mnt/nfs/home/b9017460/finder/example/star_index_without_transcriptome genomeLoad Remove

Final effective command line: STAR --runThreadN 30 --genomeDir /mnt/nfs/home/b9017460/finder/example/star_index_without_transcriptome --genomeLoad Remove Number of fastq files for each mate = 1 Finished loading and checking parameters Reading genome generation parameters:

STAR --runMode genomeGenerate --runThreadN 30 --genomeDir star_index_without_transcriptome --genomeFastaFiles Arabidopsis_thaliana.TAIR10.dna_sm.toplevel.fa --genomeSAindexNbases 12 GstrandBit=32 versionGenome 2.7.4a ~RE-DEFINED genomeType Full ~RE-DEFINED genomeFastaFiles Arabidopsis_thaliana.TAIR10.dna_sm.toplevel.fa ~RE-DEFINED genomeSAindexNbases 12 ~RE-DEFINED genomeChrBinNbits 18 ~RE-DEFINED genomeSAsparseD 1 ~RE-DEFINED genomeTransformType None ~RE-DEFINED genomeTransformVCF - ~RE-DEFINED sjdbOverhang 0 ~RE-DEFINED sjdbFileChrStartEnd - ~RE-DEFINED sjdbGTFfile - ~RE-DEFINED sjdbGTFchrPrefix - ~RE-DEFINED sjdbGTFfeatureExon exon ~RE-DEFINED sjdbGTFtagExonParentTranscripttranscript_id ~RE-DEFINED sjdbGTFtagExonParentGene gene_id ~RE-DEFINED sjdbInsertSave Basic ~RE-DEFINED genomeFileSizes 120586240 985722733 ~RE-DEFINED Genome version is compatible with current STAR Number of real (reference) chromosomes= 7 1 1 30427671 0 2 2 19698289 30670848 3 3 23459830 50593792 4 4 18585056 74186752 5 5 26975502 92798976 6 Mt 366924 119799808 7 Pt 154478 120324096 Started loading the genome: Fri Jun 18 10:22:22 2021

Genome: size given as a parameter = 120586240 SA: size given as a parameter = 985722733 SAindex: size given as a parameter = 1 Read from SAindex: pGe.gSAindexNbases=12 nSAi=22369620 nGenome=120586240; nSAbyte=985722733 GstrandBit=32 SA number of indices=238963086

EXITING: Did not find the genome in memory, did not remove any genomes from shared memory

Jun 18 10:22:22 ...... FATAL ERROR, exiting

Please help. I have tried adding the location but still not working :( I have been trying to annotate a genome for a very long time now but keep encountering errors.

Best, Lenshina

[sagnikbanerjee15]

Hi Lenshina,

Thank you for your interest in finder. The "error" that you see is not actually an error. Before finder kicks off the SRA data download, it will attempt to remove any lingering STAR images on the RAM. When the error says "EXITING: Did not find the genome in memory, did not remove any genomes from shared memory" it means that it cannot find the image in the RAM. Please ignore the message and let finder run. Let me know if you encounter any further issues.

Thank you.

[lenshinaagbor]

Hi Sagnik,

Thanks for your response. Indeed, Finder ran with the example data and my data but I still ended up with an error message and empty folders. I am using our cluster and cannot say what I'm not doing right and it is very frustrating. In brief, I am trying to annotate the genome of Trichomonas tenax (link to genome: https://sra-download.ncbi.nlm.nih.gov/traces/wgs01/wgs_aux/OC/TD/OCTD01/OCTD01.1.fsa_nt.gz) (RNAseq data; PRJNA340083, SRR4063857, 1 ILLUMINA (Illumina HiSeq 1500) run)) (link to annotated protein sequences of related species;https://trichdb.org/common/downloads/Current_Release/TvaginalisG3/fasta/data/TrichDB-52_TvaginalisG3_AnnotatedProteins.fasta).

The error files I am getting are as follows; braker error; ERROR in file /mnt/storage/nobackup/b9017460/finder/dep/BRAKER/scripts/braker.pl at line 6645 Failed to execute: perl /mnt/storage/nobackup/b9017460/finder/dep/gmes_linux_64/gmes_petap.pl --verbose --sequence=/mnt/storage/nobackup/b9017460/finder/Tenax/FINDER_test_tenax/braker/genome.fa --ET=/mnt/storage/nobackup/b9017460/finder/Tenax/FINDER_test_tenax/braker/genemark_hintsfile.gff --cores=30 --gc_donor 0.001 --soft_mask auto 1>/mnt/storage/nobackup/b9017460/finder/Tenax/FINDER_test_tenax/braker/GeneMark-ET.stdout 2>/mnt/storage/nobackup/b9017460/finder/Tenax/FINDER_test_tenax/braker/errors/GeneMark-ET.stderr Failed to execute: perl /mnt/storage/nobackup/b9017460/finder/dep/gmes_linux_64/gmes_petap.pl --verbose --sequence=/mnt/storage/nobackup/b9017460/finder/Tenax/FINDER_test_tenax/braker/genome.fa --ET=/mnt/storage/nobackup/b9017460/finder/Tenax/FINDER_test_tenax/braker/genemark_hintsfile.gff --cores=30 --gc_donor 0.001 --soft_mask auto 1>/mnt/storage/nobackup/b9017460/finder/Tenax/FINDER_test_tenax/braker/GeneMark-ET.stdout 2>/mnt/storage/nobackup/b9017460/finder/Tenax/FINDER_test_tenax/braker/errors/GeneMark-ET.stderr The most common problem is an expired or not present file ~/.gmkey!

• The final_GTF_files folder is empty
• The progress log 2021-06-21 14:05:32,085 - finder - INFO - Software paths have been set 2021-06-21 14:05:32,709 - finder - INFO - validateCommandLineArguments execution successful 2021-06-21 14:05:32,711 - finder - INFO - Metadata information created 2021-06-21 14:05:32,711 - finder - INFO - readMetaDataFile execution successful 2021-06-21 14:05:32,712 - finder - INFO - expandGzippedFiles execution successful 2021-06-21 14:05:32,713 - finder - INFO - Starting FINDER from None checkpoint 2021-06-21 14:05:32,714 - finder - INFO - Program params - Namespace(addUTR=False, checkpoint=None, compressed_data_files=None, cpu='30', error_corrected_raw_data='/nobackup/b9017460/finder/Tenax/FINDER_test_tenax/raw_data_error_corrected', exonerate_gff3=None, files_for_ncrna={'mature_ATGC': '/mnt/storage/nobackup/b9017460/finder/dep/mature_ATGC.fa'}, final_GTF_files='/nobackup/b9017460/finder/Tenax/FINDER_test_tenax/final_GTF_files', genome='/nobackup/b9017460/finder/Tenax/tenax.fa', genome_dir_olego='olego_index', genome_dir_star='/nobackup/b9017460/finder/Tenax/star_index_without_transcriptome', indices='/nobackup/b9017460/finder/Tenax/FINDER_test_tenax/indices', md=None, metadatafile='T_tenax_metadata.csv', mrna_md={'TRANSCRIPTOMIC': {'SRR4063857': {'bioproject': 'PRJNA340083', 'condition': 'TRANSCRIPTOMIC', 'Date': '2017-09-01', 'Ended': 'SE', 'desc': 'cDNA;Illumina HiSeq 1500', 'read_length': '100', 'error_corrected': 0, 'location_directory': '/nobackup/b9017460/finder/Tenax/FINDER_test_tenax/raw_data_downloaded_from_NCBI', 'downloaded_from_NCBI': 1}}}, no_cleanup=True, output_assemblies_psiclass_terminal_exon_length_modified='/nobackup/b9017460/finder/Tenax/FINDER_test_tenax/assemblies_psiclass_modified', output_braker='/nobackup/b9017460/finder/Tenax/FINDER_test_tenax/braker', output_directory='/nobackup/b9017460/finder/Tenax/FINDER_test_tenax', output_fasta_N_removed='/nobackup/b9017460/finder/Tenax/FINDER_test_tenax/raw_fasta_N_removed', output_rcorrector=None, output_sample_fastq=None, output_star='/nobackup/b9017460/finder/Tenax/FINDER_test_tenax/alignments', paired_end_adapterfile=None, perform_post_completion_data_cleanup=False, preserve_raw_input_data=True, protein='/nobackup/b9017460/finder/Tenax/new_file.fasta', raw_data_downloaded_from_NCBI='/nobackup/b9017460/finder/Tenax/FINDER_test_tenax/raw_data_downloaded_from_NCBI', record_time={}, run_tests=False, single_end_adapterfile=None, skip_cpd=False, smrna_md={}, softwares={'psiclass': '/mnt/storage/nobackup/b9017460/finder/dep/psiclass_terminal_exon_length_modified/psiclass', 'junc': '/mnt/storage/nobackup/b9017460/finder/dep/psiclass_terminal_exon_length_modified//junc', 'subexon-info': '/mnt/storage/nobackup/b9017460/finder/dep/psiclass_terminal_exon_length_modified//subexon-info', 'addXS': '/mnt/storage/nobackup/b9017460/finder/dep/psiclass_terminal_exon_length_modified//addXS', 'fastq-sample': '/mnt/storage/nobackup/b9017460/finder/dep/fastq-tools-0.8/scripts/fastq-sample', 'download_and_dump_fastq_from_SRA': '/mnt/storage/nobackup/b9017460/finder/dep/../utils/downloadAndDumpFastqFromSRA.py', 'transferGenomicNucleotideCountsToTranscriptome': '/mnt/storage/nobackup/b9017460/finder/dep/../scripts/transferGenomicNucleotideCountsToTranscriptome.py', 'find_exonic_troughs': '/mnt/storage/nobackup/b9017460/finder/dep/../scripts/find_exonic_troughs.R', 'olego': '/mnt/storage/nobackup/b9017460/finder/dep/olego/olego', 'olegoindex': '/mnt/storage/nobackup/b9017460/finder/dep/olego/olegoindex', 'mergePEsam.pl': '/mnt/storage/nobackup/b9017460/finder/dep/olego/mergePEsam.pl', 'xa2multi': '/mnt/storage/nobackup/b9017460/finder/dep/olego/xa2multi.pl', 'gmst': '/mnt/storage/nobackup/b9017460/finder/dep/gmst.pl', 'prodigal': '/mnt/storage/nobackup/b9017460/finder/dep/Prodigal/prodigal', 'canon-gff3': '/mnt/storage/nobackup/b9017460/finder/dep/canon-gff3', 'convert_exonerate_gff_to_gtf': '/mnt/storage/nobackup/b9017460/finder/dep/../utils/convert_exonerate_gff_to_gtf.py', 'augustus_main_dir': '/mnt/storage/nobackup/b9017460/finder/dep/Augustus', 'braker': '/mnt/storage/nobackup/b9017460/finder/dep/BRAKER/scripts/braker.pl', 'GENEMARK_PATH': '/mnt/storage/nobackup/b9017460/finder/dep/gmes_linux_64', 'AUGUSTUS_CONFIG_PATH': '/nobackup/b9017460/finder/Tenax/FINDER_test_tenax/braker/Augustus/config', 'AUGUSTUS_BIN_PATH': '/nobackup/b9017460/finder/Tenax/FINDER_test_tenax/braker/Augustus/bin', 'AUGUSTUS_SCRIPTS_PATH': '/nobackup/b9017460/finder/Tenax/FINDER_test_tenax/braker/Augustus/scripts', 'GUSHR_PATH': '/mnt/storage/nobackup/b9017460/finder/dep/GUSHR'}, space_saved=None, temp_dir='/nobackup/b9017460/finder/Tenax/FINDER_test_tenax/temp', total_space=None, verbose=1)2021-06-21 14:05:32,714 - finder - INFO - Started processing data for TRANSCRIPTOMIC 2021-06-21 14:05:32,737 - finder - INFO - Information collection about alignments completed 2021-06-21 14:05:32,738 - finder - INFO - Skipping the regeneration of assemblies with PsiCLASS 2021-06-21 14:05:32,739 - finder - INFO - Missed tissue specific transcripts added to gene annotations 2021-06-21 14:05:32,740 - finder - INFO - Redundant transcripts removed from gene annotations 2021-06-21 14:05:32,741 - finder - INFO - Transcripts connecting two other transcripts in opposite direction fixed 2021-06-21 14:05:32,742 - finder - INFO - Generation of transcriptomic counts completed 2021-06-21 14:05:32,742 - finder - INFO - Fix overlapping and merged transcripts completed 2021-06-21 14:05:32,744 - finder - INFO - Redundant transcripts removed from gene annotations 2021-06-21 14:05:32,745 - finder - INFO - Closely placed transcripts with very similar expression level merged 2021-06-21 14:05:32,825 - finder - INFO - Transcripts splited on intron information 2021-06-21 14:05:32,825 - finder - INFO - Redundant transcripts removed from gene annotations 2021-06-21 14:05:34,128 - finder - INFO - Associating transcripts to conditions completed 2021-06-21 14:05:34,129 - finder - INFO - orchestrateGeneModelPrediction execution successful 2021-06-21 14:05:39,567 - finder - INFO - Running command - cp -r /mnt/storage/nobackup/b9017460/finder/dep/Augustus /nobackup/b9017460/finder/Tenax/FINDER_test_tenax/braker/ 2021-06-21 14:05:39,568 - finder - INFO - BRAKER run started 2021-06-21 14:05:39,568 - finder - INFO - Running BRAKER2 - /mnt/storage/nobackup/b9017460/finder/dep/BRAKER/scripts/braker.pl --GENEMARK_PATH=/mnt/storage/nobackup/b9017460/finder/dep/gmes_linux_64 --AUGUSTUS_CONFIG_PATH=/nobackup/b9017460/finder/Tenax/FINDER_test_tenax/braker/Augustus/config --AUGUSTUS_BIN_PATH=/nobackup/b9017460/finder/Tenax/FINDER_test_tenax/braker/Augustus/bin --AUGUSTUS_SCRIPTS_PATH=/nobackup/b9017460/finder/Tenax/FINDER_test_tenax/braker/Augustus/scripts --GUSHR_PATH=/mnt/storage/nobackup/b9017460/finder/dep/GUSHR --softmasking --bam=/nobackup/b9017460/finder/Tenax/FINDER_test_tenax/alignments/SRR4063857_for_psiclass.bam, --genome=/nobackup/b9017460/finder/Tenax/FINDER_test_tenax/temp/genome_for_braker.fa --cores=30 --workingdir=/nobackup/b9017460/finder/Tenax/FINDER_test_tenax/braker --overwrite --gff3 > /nobackup/b9017460/finder/Tenax/FINDER_test_tenax/braker.output 2> /nobackup/b9017460/finder/Tenax/FINDER_test_tenax/braker.error 2021-06-21 14:08:21,997 - finder - INFO - Running command - gffread -E /nobackup/b9017460/finder/Tenax/FINDER_test_tenax/braker/braker.gff3 -T -o /nobackup/b9017460/finder/Tenax/FINDER_test_tenax/braker/braker.gtf 2021-06-21 14:08:22,020 - finder - INFO - BRAKER run completed 2021-06-21 14:08:22,573 - finder - INFO - Finding CDS completed

The error file;

EXITING: Did not find the genome in memory, did not remove any genomes from shared memory Jun 20 07:08:08 ...... FATAL ERROR, exiting EXITING: Did not find the genome in memory, did not remove any genomes from shared memory Jun 20 09:29:58 ...... FATAL ERROR, exiting EXITING: Did not find the genome in memory, did not remove any genomes from shared memory Jun 20 09:30:45 ...... FATAL ERROR, exiting /mnt/nfs/home/b9017460/finder/scripts/fixOverlappingAndMergedTranscripts.py:347: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray coverage_info[transcript_id]["bed_cov"]=np.array(temp) rm: cannot remove ‘/nobackup/b9017460/finder/Tenax/FINDER_test_tenax/assemblies_psiclassmodified/combined/outputfileforCPD*’: No such file or directory Command line was: gffread -E /nobackup/b9017460/finder/Tenax/FINDER_test_tenax/braker/braker.gff3 -T -o /nobackup/b9017460/finder/Tenax/FINDER_test_tenax/braker/braker.gtf Error: cannot open input file /nobackup/b9017460/finder/Tenax/FINDER_test_tenax/braker/braker.gff3! Traceback (most recent call last): File "/mnt/nfs/home/b9017460/finder/finder", line 668, in main() File "/mnt/nfs/home/b9017460/finder/finder", line 649, in main removeSpuriousTranscriptsBasedOnCDS(options,logger_proxy,logging_mutex) File "/mnt/nfs/home/b9017460/finder/scripts/removeSpuriousTranscriptsBasedOnCDS.py", line 53, in removeSpuriousTranscriptsBasedOnCDS all_transcripts_info_gene_to_transcripts[gene_id][transcript_id]["confidence"]="repeat_region" KeyError: 'OCTD01000001.1.1'

Please, can you help?

Thanks, Lenshina

[sagnikbanerjee15]

Hello Lenshina,

I completely understand how frustrating it can be to encounter issues like this. There seems to be an issue with braker. Sometimes braker fails on small genomes and fragmented chromosomes. Thank you for providing the data you used for annotation. I will rerun it on my end and see if I can replicate the error. I might have to include an option to skip the braker run for certain genomes. Please hang tight we will get through this!

Thank you.

[lenshinaagbor]

Thank you very much :)