search

TrinityCTAT / ctat-genome-lib-builder

Software used by Trinity CTAT for building CTAT Genome Libs, resource databases shared by Trinity CTAT components
BSD 3-Clause "New" or "Revised" License
10 stars 2 forks source link

No file or directory when running STAR - Rat novergicus library #10

Open GACGAMA opened 2 years ago

GACGAMA commented 2 years ago

I downloaded the latest version of starfusion singularity image

star-fusion.v1.11.1.simg

Then used the following command to generate a Rat novergicus library with the latest ensembl references and gtf for this species: singularity exec -e /home/janete-cerutti/starfusion/singularity/star-fusion.v1.11.1.simg /usr/local/src/STAR-Fusion/ctat-genome-lib-builder/prep_genome_lib.pl --genome_fa /home/janete-cerutti/starfusion/singularity/references/ensembl.mRatBN7.2.primary.assembly.fa --gtf /home/janete-cerutti/starfusion/singularity/references/Rattus_norvegicus.mRatBN7.2.108.gtf --pfam_db current --dfam_db rat

Which outputed the following error:

-found STAR at /usr/local/bin/STAR

-found makeblastdb at /usr/local/bin/makeblastdb

-found blastn at /usr/local/bin/blastn

-found dfamscan.pl at /usr/local/bin/dfamscan.pl

-found nhmmscan at /usr/local/bin/nhmmscan

-found hmmscan at /usr/local/bin/hmmscan

* Running CMD: wget ftp://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/Pfam-A.hmm.gz
--2022-11-01 13:34:22--  ftp://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/Pfam-A.hmm.gz
           => 'Pfam-A.hmm.gz'
Resolving ftp.ebi.ac.uk (ftp.ebi.ac.uk)... 193.62.193.138
Connecting to ftp.ebi.ac.uk (ftp.ebi.ac.uk)|193.62.193.138|:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done.    ==> PWD ... done.
==> TYPE I ... done.  ==> CWD (1) /pub/databases/Pfam/current_release ... done.
==> SIZE Pfam-A.hmm.gz ... 293000230
==> PASV ... done.    ==> RETR Pfam-A.hmm.gz ... done.
Length: 293000230 (279M) (unauthoritative)

Pfam-A.hmm.gz       100%[===================>] 279.43M  4.50MB/s    in 55s     

2022-11-01 13:35:19 (5.11 MB/s) - 'Pfam-A.hmm.gz' saved [293000230]

* Running CMD: gunzip Pfam-A.hmm.gz
* Running CMD: hmmpress Pfam-A.hmm
Working...    done.
Pressed and indexed 19632 HMMs (19632 names and 19632 accessions).
Models pressed into binary file:   Pfam-A.hmm.h3m
SSI index for binary model file:   Pfam-A.hmm.h3i
Profiles (MSV part) pressed into:  Pfam-A.hmm.h3f
Profiles (remainder) pressed into: Pfam-A.hmm.h3p
* Running CMD: cp /home/janete-cerutti/starfusion/singularity/references/ensembl.mRatBN7.2.primary.assembly.fa /home/janete-cerutti/starfusion/singularity/ctat_genome_lib_build_dir/ref_genome.fa
* Running CMD: samtools faidx /home/janete-cerutti/starfusion/singularity/ctat_genome_lib_build_dir/ref_genome.fa
* Running CMD: makeblastdb -in /home/janete-cerutti/starfusion/singularity/ctat_genome_lib_build_dir/ref_genome.fa -dbtype nucl

Building a new DB, current time: 11/01/2022 13:36:19
New DB name:   /home/janete-cerutti/starfusion/singularity/ctat_genome_lib_build_dir/ref_genome.fa
New DB title:  /home/janete-cerutti/starfusion/singularity/ctat_genome_lib_build_dir/ref_genome.fa
Sequence type: Nucleotide
Keep MBits: T
Maximum file size: 1000000000B
Adding sequences from FASTA; added 23 sequences in 31.9478 seconds.
* Running CMD: cp /home/janete-cerutti/starfusion/singularity/references/Rattus_norvegicus.mRatBN7.2.108.gtf /home/janete-cerutti/starfusion/singularity/ctat_genome_lib_build_dir/ref_annot.gtf
* Running CMD: bash -c " set -euxo pipefail; /usr/local/src/STAR-Fusion/ctat-genome-lib-builder/util/gtf_to_exon_gene_records.pl /home/janete-cerutti/starfusion/singularity/ctat_genome_lib_build_dir/ref_annot.gtf  | sort -k 1,1 -k4,4g -k5,5g | uniq  > /home/janete-cerutti/starfusion/singularity/ctat_genome_lib_build_dir/ref_annot.gtf.mini.sortu " 
+ /usr/local/src/STAR-Fusion/ctat-genome-lib-builder/util/gtf_to_exon_gene_records.pl /home/janete-cerutti/starfusion/singularity/ctat_genome_lib_build_dir/ref_annot.gtf
+ sort -k 1,1 -k4,4g -k5,5g
+ uniq
* Running CMD: STAR --runThreadN 4 --runMode genomeGenerate --genomeDir /home/janete-cerutti/starfusion/singularity/ctat_genome_lib_build_dir/ref_genome.fa.star.idx   --genomeFastaFiles /home/janete-cerutti/starfusion/singularity/references/ensembl.mRatBN7.2.primary.assembly.fa  --limitGenomeGenerateRAM 40419136213  --genomeChrBinNbits 16  --sjdbGTFfile /home/janete-cerutti/starfusion/singularity/references/Rattus_norvegicus.mRatBN7.2.108.gtf  --sjdbOverhang 150 
Nov 01 13:36:56 ..... started STAR run
Nov 01 13:36:56 ... starting to generate Genome files
Nov 01 13:37:43 ..... processing annotations GTF
Nov 01 13:38:02 ... starting to sort Suffix Array. This may take a long time...
Nov 01 13:38:17 ... sorting Suffix Array chunks and saving them to disk...
Error, cmd: STAR --runThreadN 4 --runMode genomeGenerate --genomeDir /home/janete-cerutti/starfusion/singularity/ctat_genome_lib_build_dir/ref_genome.fa.star.idx   --genomeFastaFiles /home/janete-cerutti/starfusion/singularity/references/ensembl.mRatBN7.2.primary.assembly.fa  --limitGenomeGenerateRAM 40419136213  --genomeChrBinNbits 16  --sjdbGTFfile /home/janete-cerutti/starfusion/singularity/references/Rattus_norvegicus.mRatBN7.2.108.gtf  --sjdbOverhang 150  died with ret 9 No such file or directory at /usr/local/src/STAR-Fusion/ctat-genome-lib-builder/lib/Pipeliner.pm line 186.
    Pipeliner::run(Pipeliner=HASH(0x560ded0b1bd0)) called at /usr/local/src/STAR-Fusion/ctat-genome-lib-builder/prep_genome_lib.pl line 460
brianjohnhaas commented 2 years ago

hi,

that step can use a lot of RAM. Do you have ~50G available to it?

On Tue, Nov 1, 2022 at 12:44 PM GACGAMA @.***> wrote:

I downloaded the latest version of starfusion singularity image

star-fusion.v1.11.1.simg

Then used the following command to generate a Rat novergicus library with the latest ensembl references and gtf for this species: singularity exec -e /home/janete-cerutti/starfusion/singularity/star-fusion.v1.11.1.simg /usr/local/src/STAR-Fusion/ctat-genome-lib-builder/prep_genome_lib.pl --genome_fa /home/janete-cerutti/starfusion/singularity/references/ensembl.mRatBN7.2.primary.assembly.fa --gtf /home/janete-cerutti/starfusion/singularity/references/Rattus_norvegicus.mRatBN7.2.108.gtf --pfam_db current --dfam_db rat

Which outputed the following error:

-found STAR at /usr/local/bin/STAR

-found makeblastdb at /usr/local/bin/makeblastdb

-found blastn at /usr/local/bin/blastn

-found dfamscan.pl at /usr/local/bin/dfamscan.pl

-found nhmmscan at /usr/local/bin/nhmmscan

-found hmmscan at /usr/local/bin/hmmscan

  • Running CMD: wget ftp://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/Pfam-A.hmm.gz --2022-11-01 ftp://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/Pfam-A.hmm.gz--2022-11-01 13:34:22-- ftp://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/Pfam-A.hmm.gz => 'Pfam-A.hmm.gz' Resolving ftp.ebi.ac.uk (ftp.ebi.ac.uk)... 193.62.193.138 Connecting to ftp.ebi.ac.uk (ftp.ebi.ac.uk)|193.62.193.138|:21... connected. Logging in as anonymous ... Logged in! ==> SYST ... done. ==> PWD ... done. ==> TYPE I ... done. ==> CWD (1) /pub/databases/Pfam/current_release ... done. ==> SIZE Pfam-A.hmm.gz ... 293000230 ==> PASV ... done. ==> RETR Pfam-A.hmm.gz ... done. Length: 293000230 (279M) (unauthoritative)

Pfam-A.hmm.gz 100%[===================>] 279.43M 4.50MB/s in 55s

2022-11-01 13:35:19 (5.11 MB/s) - 'Pfam-A.hmm.gz' saved [293000230]

  • Running CMD: gunzip Pfam-A.hmm.gz
  • Running CMD: hmmpress Pfam-A.hmm Working... done. Pressed and indexed 19632 HMMs (19632 names and 19632 accessions). Models pressed into binary file: Pfam-A.hmm.h3m SSI index for binary model file: Pfam-A.hmm.h3i Profiles (MSV part) pressed into: Pfam-A.hmm.h3f Profiles (remainder) pressed into: Pfam-A.hmm.h3p
  • Running CMD: cp /home/janete-cerutti/starfusion/singularity/references/ensembl.mRatBN7.2.primary.assembly.fa /home/janete-cerutti/starfusion/singularity/ctat_genome_lib_build_dir/ref_genome.fa
  • Running CMD: samtools faidx /home/janete-cerutti/starfusion/singularity/ctat_genome_lib_build_dir/ref_genome.fa
  • Running CMD: makeblastdb -in /home/janete-cerutti/starfusion/singularity/ctat_genome_lib_build_dir/ref_genome.fa -dbtype nucl

Building a new DB, current time: 11/01/2022 13:36:19 New DB name: /home/janete-cerutti/starfusion/singularity/ctat_genome_lib_build_dir/ref_genome.fa New DB title: /home/janete-cerutti/starfusion/singularity/ctat_genome_lib_build_dir/ref_genome.fa Sequence type: Nucleotide Keep MBits: T Maximum file size: 1000000000B Adding sequences from FASTA; added 23 sequences in 31.9478 seconds.

  • Running CMD: cp /home/janete-cerutti/starfusion/singularity/references/Rattus_norvegicus.mRatBN7.2.108.gtf /home/janete-cerutti/starfusion/singularity/ctat_genome_lib_build_dir/ref_annot.gtf
  • Running CMD: bash -c " set -euxo pipefail; /usr/local/src/STAR-Fusion/ctat-genome-lib-builder/util/gtf_to_exon_gene_records.pl /home/janete-cerutti/starfusion/singularity/ctat_genome_lib_build_dir/ref_annot.gtf | sort -k 1,1 -k4,4g -k5,5g | uniq > /home/janete-cerutti/starfusion/singularity/ctat_genome_lib_build_dir/ref_annot.gtf.mini.sortu "
  • /usr/local/src/STAR-Fusion/ctat-genome-lib-builder/util/gtf_to_exon_gene_records.pl /home/janete-cerutti/starfusion/singularity/ctat_genome_lib_build_dir/ref_annot.gtf
  • sort -k 1,1 -k4,4g -k5,5g
  • uniq
  • Running CMD: STAR --runThreadN 4 --runMode genomeGenerate --genomeDir /home/janete-cerutti/starfusion/singularity/ctat_genome_lib_build_dir/ref_genome.fa.star.idx --genomeFastaFiles /home/janete-cerutti/starfusion/singularity/references/ensembl.mRatBN7.2.primary.assembly.fa --limitGenomeGenerateRAM 40419136213 --genomeChrBinNbits 16 --sjdbGTFfile /home/janete-cerutti/starfusion/singularity/references/Rattus_norvegicus.mRatBN7.2.108.gtf --sjdbOverhang 150 Nov 01 13:36:56 ..... started STAR run Nov 01 13:36:56 ... starting to generate Genome files Nov 01 13:37:43 ..... processing annotations GTF Nov 01 13:38:02 ... starting to sort Suffix Array. This may take a long time... Nov 01 13:38:17 ... sorting Suffix Array chunks and saving them to disk... Error, cmd: STAR --runThreadN 4 --runMode genomeGenerate --genomeDir /home/janete-cerutti/starfusion/singularity/ctat_genome_lib_build_dir/ref_genome.fa.star.idx --genomeFastaFiles /home/janete-cerutti/starfusion/singularity/references/ensembl.mRatBN7.2.primary.assembly.fa --limitGenomeGenerateRAM 40419136213 --genomeChrBinNbits 16 --sjdbGTFfile /home/janete-cerutti/starfusion/singularity/references/Rattus_norvegicus.mRatBN7.2.108.gtf --sjdbOverhang 150 died with ret 9 No such file or directory at /usr/local/src/STAR-Fusion/ctat-genome-lib-builder/lib/Pipeliner.pm line 186. Pipeliner::run(Pipeliner=HASH(0x560ded0b1bd0)) called at /usr/local/src/STAR-Fusion/ctat-genome-lib-builder/prep_genome_lib.pl line 460

— Reply to this email directly, view it on GitHub https://github.com/NCIP/ctat-genome-lib-builder/issues/10, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABZRKX4ZOWTIWSJUVOM2OIDWGFCIBANCNFSM6AAAAAARUHOJZQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

--

Brian J. Haas The Broad Institute http://broadinstitute.org/~bhaas http://broad.mit.edu/~bhaas