search

AuReMe / metage2metabo

From annotated genomes to metabolic screening in large scale microbiotas
https://metage2metabo.readthedocs.io
GNU Lesser General Public License v3.0
50 stars 7 forks source link

Error for teste with PathoLogic subprocess, return code: 255 #13

Closed Lucas-Maciel closed 2 years ago

Lucas-Maciel commented 3 years ago

Hi,

I'm trying to run your pipeline but I had an error. Do you have any idea how can I solve this? I'm leaving here the input and outputs.Thank you very much m2m_test.zip ` ######### Running metabolic network reconstruction with Pathway Tools ######### Check and delete unfinished builds of Pathway Tools.


Checking inputs for teste: missing flat_files_creation.lisp; genetic-elements.dat; organism-params.dat. Inputs file created for teste.
----------End of creation of input data from Genbank/GFF/PF: 0.41s----------
~~~~~~~~~~Inference on the data~~~~~~~~~~
pathway-tools -no-web-cel-overview -no-cel-overview -no-patch-download -disable-metadata-saving -nologfile -patho gbk/teste
!!!!!!!!!!!!!!!!!----------------------------------------!!!!!!!!!!!!!!!!!
Error for teste with PathoLogic subprocess, return code: 255
=== Error in pathologic.log for gbk/teste===
    Error from the pathologic.log file: gbk/teste/pathologic.log
    batch-pathologic: A fatal error occurred for gbk/teste/ version 1.0.

    See pathologic log file gbk/teste/pathologic.log for more details.

    03-Mar-2021  00:31:11 Fatal error: file #P"gbk/teste/migs.dat" does not exist: No such file or directory [errno=2].

    Evaluation stack:

     ->(TPL::ZOOM-COMMAND :FROM-READ-EVAL-PRINT-LOOP NIL ...)

       (SYS::..RUNTIME-OPERATION "applyn" :UNKNOWN-ARGS)

       (TPL:DO-COMMAND "zoom" :FROM-READ-EVAL-PRINT-LOOP ...)

=== Pathway Tools log ===
    ;; Optimization settings: safety 3, space 1, speed 1, debug 3.

    ;; For a complete description of all compiler switches given the

    ;; current optimization settings evaluate (EXPLAIN-COMPILER-SETTINGS).

    *** This Pathway Tools executable built on Thu Jan 14, 2021 at 17:34:28. ***

    [oot::acache-connect: Note that another Ocelot KB named NCBI-TAXONOMY already exists (#<OCELOT-FILE-KB

                                                                                            NCBI-TAXONOMY NIL

                                                                                            @

                                                                                            #x10044b029d2>); creating new KB named NCBI-TAXONOMY]

    [Opened acache database /shared/software/apps/pathway-tools/24.5/aic-export/pathway-tools/ocelot-acache/, which contains 1 ocelot KBs]

    [Reading Pathway Tools init file "/raeslab/scratch/pathway-tools/ptools-local/ptools-init.dat" ]

    [Scanning PGDB directories in   /raeslab/scratch/pathway-tools/ptools-local/pgdbs/user/   3 total PGDBs have now been found]

    [Scanning PGDB directories in   /raeslab/scratch/pathway-tools/ptools-local/pgdbs/registry/   3 total PGDBs have now been found]

    [Loading Ocelot KB from /shared/software/apps/pathway-tools/24.5/aic-export/pgdbs/biocyc/PGDB-METADATA.ocelot

     KB name=PGDB-METADATA, format=V1-SEXPR

    Warning: Skipping load of frame ECOLI-0 which already exists in PGDB-METADATA

    Warning: Skipping load of frame META-NIL which already exists in PGDB-METADATA

     0 frames loaded]

    [Scanning PGDB directories in   /shared/software/apps/pathway-tools/24.5/aic-export/pgdbs/biocyc/   3 total PGDBs have now been found]

    ====== SCIP, package: #<The SCIP

                            package>, #+-scip: SCIP, (fboundp 'load-scip-lib-at-execution-time): #<Function LOAD-SCIP-LIB-AT-EXECUTION-TIME>.

    Loading SCIP Library at Start Up (Runtime?: PTOOLS-RUNTIME)...

    Loading scip lib from scip-interface: /shared/software/apps/pathway-tools/24.5/aic-export/pathway-tools/ptools/24.5/exe/libScipAll.so

            ::::::::  Begin Batch PathoLogic for gbk/teste/  ::::::::

    [Redirecting standard-output and error-output to gbk/teste/pathologic.log]

    20 40 60 80 100 120 140 20 40 60 80 100 Copying file gbk/teste/genetic-elements.dat

        to       /raeslab/scratch/pathway-tools/ptools-local/pgdbs/user/testecyc/1.0/input/genetic-elements.dat

    Copying file /raeslab/scratch/lucmac/bin/emapper_to_gbk-0.0.7/m2m_test/gbk/teste/teste.gbk

        to       /raeslab/scratch/pathway-tools/ptools-local/pgdbs/user/testecyc/1.0/input/teste.gbk

    [Indexed TESTEBASE class People of 21 frames yielding hash table of 53 entries]

    [Indexed TESTEBASE class Growth-Media of 1 frames yielding hash table of 2 entries]

    [Indexed TESTEBASE class Extragenic-Sites of 3 frames yielding hash table of 4 entries]

    [Indexed TESTEBASE class Organisms of 22 frames yielding hash table of 101 entries]

    [Indexed TESTEBASE class Transcription-Units of 1 frames yielding hash table of 1 entries]

    [Indexed TESTEBASE class Gene-Ontology-Terms of 3 frames yielding hash table of 14 entries]

    [Indexed TESTEBASE class DNAs of 7 frames yielding hash table of 11 entries]

    [Indexed TESTEBASE class Reactions of 40 frames yielding hash table of 51 entries]

    [Indexed TESTEBASE class Pathways of 672 frames yielding hash table of 1145 entries]

    [Indexed TESTEBASE class Compounds of 4662 frames yielding hash table of 12745 entries]

    [Indexed TESTEBASE class Polynucleotides of 281 frames yielding hash table of 637 entries]

    [Indexed TESTEBASE class Proteins of 168 frames yielding hash table of 451 entries]

    [Indexed TESTEBASE class Enzymatic-Reactions of 1 frames yielding hash table of 0 entries]

    [Indexed TESTEBASE class RNAs of 248 frames yielding hash table of 572 entries]

    [Indexed TESTEBASE class All-Genes of 10 frames yielding hash table of 12 entries]

    100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000 2100 2200 2300 2400 2500 2600 2700 2800 2900 3000 3100 3200 3300 3400 3500 3600 3700 3800 3900 4000 4100 4200 4300 4400 4500 4600 4700 4800 4900 5000 5100 5200 5300 5400 5500 5600 5700 5800 5900 6000 6100 6200 6300 6400 6500 6600 6700 6800 6900 7000 7100 7200 7300 7400 7500 7600 7700 7800 7900 8000 8100 8200 8300 8400 8500 8600 8700 8800 8900 9000 9100 9200 9300 9400 9500 9600 9700 9800 9900 10000 10100 10200 10300 10400 10500 10600 10700 10800 10900 11000 11100 11200 11300 11400 11500 11600 11700 
!!!!!!!!!!!!!!!!!----------------------------------------!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!----------------------------------------!!!!!!!!!!!!!!!!!
=== Error in pathologic.log for gbk/teste===
    Error from the pathologic.log file: gbk/teste/pathologic.log
    batch-pathologic: A fatal error occurred for gbk/teste/ version 1.0.

    See pathologic log file gbk/teste/pathologic.log for more details.

    03-Mar-2021  00:31:11 Fatal error: file #P"gbk/teste/migs.dat" does not exist: No such file or directory [errno=2].

    Evaluation stack:

     ->(TPL::ZOOM-COMMAND :FROM-READ-EVAL-PRINT-LOOP NIL ...)

       (SYS::..RUNTIME-OPERATION "applyn" :UNKNOWN-ARGS)

       (TPL:DO-COMMAND "zoom" :FROM-READ-EVAL-PRINT-LOOP ...)

=== Pathway Tools log ===
    ;; Optimization settings: safety 3, space 1, speed 1, debug 3.

    ;; For a complete description of all compiler switches given the

    ;; current optimization settings evaluate (EXPLAIN-COMPILER-SETTINGS).

    *** This Pathway Tools executable built on Thu Jan 14, 2021 at 17:34:28. ***

    [oot::acache-connect: Note that another Ocelot KB named NCBI-TAXONOMY already exists (#<OCELOT-FILE-KB

                                                                                            NCBI-TAXONOMY NIL

                                                                                            @

                                                                                            #x10044b029d2>); creating new KB named NCBI-TAXONOMY]

    [Opened acache database /shared/software/apps/pathway-tools/24.5/aic-export/pathway-tools/ocelot-acache/, which contains 1 ocelot KBs]

    [Reading Pathway Tools init file "/raeslab/scratch/pathway-tools/ptools-local/ptools-init.dat" ]

    [Scanning PGDB directories in   /raeslab/scratch/pathway-tools/ptools-local/pgdbs/user/   3 total PGDBs have now been found]

    [Scanning PGDB directories in   /raeslab/scratch/pathway-tools/ptools-local/pgdbs/registry/   3 total PGDBs have now been found]

    [Loading Ocelot KB from /shared/software/apps/pathway-tools/24.5/aic-export/pgdbs/biocyc/PGDB-METADATA.ocelot

     KB name=PGDB-METADATA, format=V1-SEXPR

    Warning: Skipping load of frame ECOLI-0 which already exists in PGDB-METADATA

    Warning: Skipping load of frame META-NIL which already exists in PGDB-METADATA

     0 frames loaded]

    [Scanning PGDB directories in   /shared/software/apps/pathway-tools/24.5/aic-export/pgdbs/biocyc/   3 total PGDBs have now been found]

    ====== SCIP, package: #<The SCIP

                            package>, #+-scip: SCIP, (fboundp 'load-scip-lib-at-execution-time): #<Function LOAD-SCIP-LIB-AT-EXECUTION-TIME>.

    Loading SCIP Library at Start Up (Runtime?: PTOOLS-RUNTIME)...

    Loading scip lib from scip-interface: /shared/software/apps/pathway-tools/24.5/aic-export/pathway-tools/ptools/24.5/exe/libScipAll.so

            ::::::::  Begin Batch PathoLogic for gbk/teste/  ::::::::

    [Redirecting standard-output and error-output to gbk/teste/pathologic.log]

    20 40 60 80 100 120 140 20 40 60 80 100 Copying file gbk/teste/genetic-elements.dat

        to       /raeslab/scratch/pathway-tools/ptools-local/pgdbs/user/testecyc/1.0/input/genetic-elements.dat

    Copying file /raeslab/scratch/lucmac/bin/emapper_to_gbk-0.0.7/m2m_test/gbk/teste/teste.gbk

        to       /raeslab/scratch/pathway-tools/ptools-local/pgdbs/user/testecyc/1.0/input/teste.gbk

    [Indexed TESTEBASE class People of 21 frames yielding hash table of 53 entries]

    [Indexed TESTEBASE class Growth-Media of 1 frames yielding hash table of 2 entries]

    [Indexed TESTEBASE class Extragenic-Sites of 3 frames yielding hash table of 4 entries]

    [Indexed TESTEBASE class Organisms of 22 frames yielding hash table of 101 entries]

    [Indexed TESTEBASE class Transcription-Units of 1 frames yielding hash table of 1 entries]

    [Indexed TESTEBASE class Gene-Ontology-Terms of 3 frames yielding hash table of 14 entries]

    [Indexed TESTEBASE class DNAs of 7 frames yielding hash table of 11 entries]

    [Indexed TESTEBASE class Reactions of 40 frames yielding hash table of 51 entries]

    [Indexed TESTEBASE class Pathways of 672 frames yielding hash table of 1145 entries]

    [Indexed TESTEBASE class Compounds of 4662 frames yielding hash table of 12745 entries]

    [Indexed TESTEBASE class Polynucleotides of 281 frames yielding hash table of 637 entries]

    [Indexed TESTEBASE class Proteins of 168 frames yielding hash table of 451 entries]

    [Indexed TESTEBASE class Enzymatic-Reactions of 1 frames yielding hash table of 0 entries]

    [Indexed TESTEBASE class RNAs of 248 frames yielding hash table of 572 entries]

    [Indexed TESTEBASE class All-Genes of 10 frames yielding hash table of 12 entries]

    100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000 2100 2200 2300 2400 2500 2600 2700 2800 2900 3000 3100 3200 3300 3400 3500 3600 3700 3800 3900 4000 4100 4200 4300 4400 4500 4600 4700 4800 4900 5000 5100 5200 5300 5400 5500 5600 5700 5800 5900 6000 6100 6200 6300 6400 6500 6600 6700 6800 6900 7000 7100 7200 7300 7400 7500 7600 7700 7800 7900 8000 8100 8200 8300 8400 8500 8600 8700 8800 8900 9000 9100 9200 9300 9400 9500 9600 9700 9800 9900 10000 10100 10200 10300 10400 10500 10600 10700 10800 10900 11000 11100 11200 11300 11400 11500 11600 11700 
!!!!!!!!!!!!!!!!!----------------------------------------!!!!!!!!!!!!!!!!!
~~~~~~~~~~Check inference~~~~~~~~~~
No log directory, it will be created.
WARNING: 1 build has failed! See the log for more information.
`
cfrioux commented 3 years ago

Hi Lucas, Thanks for reporting the error and sharing the logs, very helpful. At first glance it seems to be an error related to Pathway Tools, could you please tell us the version you use? In addition please share the versions of mpwt and metage2metabo you have installed. Cheers

Lucas-Maciel commented 3 years ago

Hi,

mpwt v0.6.0 pathway-tools 24.5 m2m 1.4.1

akoth9 commented 3 years ago

This issue has been fixed. Please remove empty migs file from the input directory.Please restart the pathway tools to load the latest patch.

Lucas-Maciel commented 3 years ago

Hi @akoth9 , how do I restart the pathway tools? I tried to run just the command pathway-tools and a few patches were downloaded. But I'm still not able to produce migs files.

04-Mar-2021 13:04:46 Fatal error: file #P"gbk/teste/migs.dat" does not exist: No such file or directory [errno=2].

ArnaudBelcour commented 3 years ago

Hi,

Thank you @akoth9 for the fix.

@Lucas-Maciel, it is strange I update Pathway Tools on my computer and it fixes the issue with migs.dat on your data.

Pathway Tools automatically downloads the last patch when you use the pathway-tools command. Just to be sure, can you use the following command:

pathway-tools -id

It will list all the patches associated to pathway-tools. And in the list, you have these lines:

...
**** The history for this executable is: ****

This image contains systems (Ptools), version (24.5 . 24.5)
 loaded from /home/ratel/pathway-tools/aic-export/pathway-tools/ptools/24.5/
 load began on Thu Jan 14, 2021 at 17:34:28 (successful completion assumed)
The following patch files are loaded:
  patch-package-sri
  p4513-operon
  p4512-operon
  p4511-migs-issue-batch-pathologic
  p4510-operon
  p4509-metacyc-pwy-org-crash
...

The p4511-migs-issue-batch-pathologic is the patch fixing the migs.dat issue. Do you see it?

Also, in what environment is your Pathway Tools installed: on a personal computer, on a cluster? One possibility that comes into my mind is when using Pathway Tools with Singularity the patches are not correctly applied due to permission issue.

Lucas-Maciel commented 3 years ago

I was finally able to run, but now I got a new one, related to my input pathologic.log

Inference statistics:
Build failed: 1
Species: teste

------------ Species: teste
batch-pathologic: A fatal error occurred for gbk/teste/ version 1.0.
See pathologic log file gbk/teste/pathologic.log for more details.
04-Mar-2021  18:14:25 Fatal error: end of requested sequence 1178 exceeds length of cached chromosome 502
Evaluation stack:

 ->(TPL::ZOOM-COMMAND :FROM-READ-EVAL-PRINT-LOOP NIL ...)
   (SYS::..RUNTIME-OPERATION "applyn" :UNKNOWN-ARGS)
   (TPL:DO-COMMAND "zoom" :FROM-READ-EVAL-PRINT-LOOP ...)
ArnaudBelcour commented 3 years ago

Good news if the migs.dat error is fixed.

For the new error:

04-Mar-2021 18:14:25 Fatal error: end of requested sequence 1178 exceeds length of cached chromosome 502 Evaluation stack:

This message is saying that a gene with a sequence of size 1178 is in a chromosome with a sequence of size 502. And this is linked to the name of the locus of some of the records of the genbank file. At some point in the file, the locus IDs are all the same ('lcl'). Pathway Tools use this as an ID but as there is multiple records with the same ID they are overwrote by the information of the next record.

For example, this record with a gene sequence of 1178:

LOCUS       lcl                     1178 bp    DNA              UNK 17-FEB-2021
DEFINITION  cellular organisms genome.
ACCESSION   lcl|NZ_ACFY01000003.1_cds_3836
VERSION     lcl|NZ_ACFY01000003.1_cds_3836
KEYWORDS    cellular organisms.
SOURCE      .
  ORGANISM  cellular organisms
            .
FEATURES             Location/Qualifiers
     source          1..1178
                     /scaffold="lcl|NZ_ACFY01000003.1_cds_3836"
                     /db_xref="taxon:131567"
     gene            2..1178
                     /locus_tag="lcl_0001"
     CDS             2..1178
                     /locus_tag="lcl_0001"
ORIGIN
        1 atgaaagaat tagagttaaa atacggatgt aatccaaacc agaaaccatc cagaatttat
       61 atggaaaacg gcgagcttcc aatcaaagtg ttaaacggaa aaccgggtta tatcaacttt
      121 ttagatgcat ttaacggctg gcagctggta agtgaattaa agaaagcaac cggacttccg
      181 gcagcaacct cttttaaaca tgtatctccg gctggagcag cagttggact tccattgtct
      241 gaggtagagc gcaaaatcta ttgggtagac gatatggacg tagagttcac accgcttgca
      301 aacgcctata tcagagccag aggcgcggac agaatgtctt ctttcggcga tttcatttcc
      361 ttatccgatg tctgtgacaa agagacagca cttgtaatca aacgagaggt gtctgatggt
      421 gtgatcgcac cgggctatac agatgaggca cttgagattt taaaggcaaa gaaaaaggaa
      481 actacaatgt gatcgagatc gatccgaatt atgtgccggc accaatcgaa cataaagaag
      541 tgtttggaat cacatttgaa cagggaagaa atgagcttgt gatcgatgag catttctttg
      601 acaacatcgt gacagagaat aaggaaatcc cggattctgc gaaaatggat cttgcaattt
      661 ccatgattac attaaaatac acacagtcca attccgtatg ctatgtcaaa ggtggacagg
      721 caatcggtat cggtgccgga cagcagtcca gaattcactg taccagactg gccggctcta
      781 aggcagataa ctggtggtta cgccagtctc cacaggtgct cggacttcag tttttagata
      841 agattggacg tgcagacaga gataatgcga ttgatctcta catcggtgag gactatatgg
      901 atgtgcttgc agatggcgca tgggagaata tttttaaagt aaaaccggaa gtgtttacaa
      961 gggaggaaaa acgcgcatgg ttagataaaa acactgacgt tgcgcttggt tccgatgcat
     1021 tcttcccatt tggagataat gtggaacgtg ctcataagag cggagtgaaa tacatcgccc
     1081 agccaggtgg ttctatccgt gatgatcatg tcattgcaac ctgtaacaaa tacggcattg
     1141 ccatggcatt caccggaatt cgcttattcc atcattaa
//

will be used with the chromosome sequence of the last record with the ID 'lcl':

LOCUS       lcl                      502 bp    DNA              UNK 17-FEB-2021
DEFINITION  cellular organisms genome.
ACCESSION   lcl|NZ_ACFY01000177.1_cds_30
VERSION     lcl|NZ_ACFY01000177.1_cds_30
KEYWORDS    cellular organisms.
SOURCE      .
  ORGANISM  cellular organisms
            .
FEATURES             Location/Qualifiers
     source          1..502
                     /scaffold="lcl|NZ_ACFY01000177.1_cds_30"
                     /db_xref="taxon:131567"
     gene            2..502
                     /locus_tag="lcl_0001"
     CDS             2..502
                     /locus_tag="lcl_0001"
ORIGIN
        1 atgtttgaag taggagaata tattgtttat ggatgtaaag gtgtatgtca ggtagaagaa
       61 atcacacata ttgatattcc tggcagtaat aaggacaggc tgtattatgt tttggctccg
      121 ttagaggaca gaaacggcaa aatatatgcc ccgaccgata atgccaaggt ggcaatgaga
      181 aaagtcatta ccagacagga ggcagagcag ctgatcgagg agatgccgca gatcgaagaa
      241 ctgtgggtag tcaatgaaaa acagcgggaa ctgcaatata aagaggcgtt aaagacctgt
      301 gatttcaggg catggatcag tattatcaag acattatatt tccgaaaaaa aagagcggat
      361 cgctcagggc aaaaaaataa catctttgga tgaacgttat ttaaaggctg cagaaaatga
      421 gctttatggc gaattgtcac tgatattggg gcttcctaag agtcagatgg aagatttcat
      481 ttgcagaaag cttgaggcgt ag
//

because they have the same locus ID. That's why, the error message says that there is a gene sequence of 1178 with a chromosome sequence of 502.

To fix this, you need to rename these locus IDs to make them different.

And I think the same idea goes for the locus_tag of the gene/CDS, which are all "lcl_0001". It should be better if they were different.

cfrioux commented 2 years ago

Hi, I suppose this issue was solved. I'll close it, do not hesitate to reopen it if needed.