search

ncbi / pgap

NCBI Prokaryotic Genome Annotation Pipeline
Other
301 stars 89 forks source link

Running the pipeline on test genome failing #230

Closed ac-simpson closed 1 year ago

ac-simpson commented 1 year ago

Problem After running the installation as described in Quick Start, running the pipeline on the MG37 test genome produces "PGAP failed, docker exited with rc = 1". Running the test genome with "--debug" output is pasted below. Getting the line "Error: (802.5) CObjectOStream: error at line 126: Seq-entry.seq.descr.E.pub.pub.E: cannot write empty choice terminate called after throwing an instance of 'ncbi::CSerialException' " I see that someone else had this issue back in July but the solution was to update the program - and as far as I can tell I am running the latest version. Thanks!

Software versions:

PGAP version 2022-10-03.build6384 is up to date.
Output will be placed in: /home/meepmeep/mg37_results
PGAP failed, docker exited with rc = 1
Printing log starting from failed job:

[2022-11-28 22:41:56] DEBUG [step pgapx_yaml_ctl] job input {
    "file:///pgap/pgap/prepare_user_input2.cwl#pgapx_yaml_ctl/ignore_all_errors": null,
    "file:///pgap/pgap/prepare_user_input2.cwl#pgapx_yaml_ctl/input": {
        "location": "file:///pgap/output/debug/tmp-outdir/b34_7ebw/submol.json",
        "basename": "submol.json",
        "nameroot": "submol",
        "nameext": ".json",
        "class": "File",
        "checksum": "sha1$884f2c965978f27fdd67960b08a33bdd9fd25b16",
        "size": 1206,
        "http://commonwl.org/cwltool#generation": 0
    },
    "file:///pgap/pgap/prepare_user_input2.cwl#pgapx_yaml_ctl/input_fasta": {
        "class": "File",
        "location": "file:///pgap/user_input/ASM2732v1.annotation.nucleotide.1.fasta",
        "size": 588482,
        "basename": "ASM2732v1.annotation.nucleotide.1.fasta",
        "nameroot": "ASM2732v1.annotation.nucleotide.1",
        "nameext": ".fasta"
    },
    "file:///pgap/pgap/prepare_user_input2.cwl#pgapx_yaml_ctl/no_internet": null,
    "file:///pgap/pgap/prepare_user_input2.cwl#pgapx_yaml_ctl/taxon_db": {
        "class": "File",
        "location": "file:///pgap/input/uniColl_path/taxonomy.sqlite3",
        "basename": "taxonomy.sqlite3",
        "size": 1072103424,
        "nameroot": "taxonomy",
        "nameext": ".sqlite3"
    }
}
[2022-11-28 22:41:56] DEBUG [step pgapx_yaml_ctl] evaluated job input to {
    "file:///pgap/pgap/prepare_user_input2.cwl#pgapx_yaml_ctl/ignore_all_errors": null,
    "file:///pgap/pgap/prepare_user_input2.cwl#pgapx_yaml_ctl/input": {
        "location": "file:///pgap/output/debug/tmp-outdir/b34_7ebw/submol.json",
        "basename": "submol.json",
        "nameroot": "submol",
        "nameext": ".json",
        "class": "File",
        "checksum": "sha1$884f2c965978f27fdd67960b08a33bdd9fd25b16",
        "size": 1206,
        "http://commonwl.org/cwltool#generation": 0
    },
    "file:///pgap/pgap/prepare_user_input2.cwl#pgapx_yaml_ctl/input_fasta": {
        "class": "File",
        "location": "file:///pgap/user_input/ASM2732v1.annotation.nucleotide.1.fasta",
        "size": 588482,
        "basename": "ASM2732v1.annotation.nucleotide.1.fasta",
        "nameroot": "ASM2732v1.annotation.nucleotide.1",
        "nameext": ".fasta"
    },
    "file:///pgap/pgap/prepare_user_input2.cwl#pgapx_yaml_ctl/no_internet": null,
    "file:///pgap/pgap/prepare_user_input2.cwl#pgapx_yaml_ctl/taxon_db": {
        "class": "File",
        "location": "file:///pgap/input/uniColl_path/taxonomy.sqlite3",
        "basename": "taxonomy.sqlite3",
        "size": 1072103424,
        "nameroot": "taxonomy",
        "nameext": ".sqlite3"
    }
}
[2022-11-28 22:41:56] INFO [step pgapx_yaml_ctl] start
[2022-11-28 22:41:56] DEBUG [job pgapx_yaml_ctl] initializing from file:///pgap/pgap/progs/pgapx_yaml_ctl.cwl as part of step pgapx_yaml_ctl
[2022-11-28 22:41:56] DEBUG [job pgapx_yaml_ctl] {
    "ignore_all_errors": null,
    "input": {
        "location": "file:///pgap/output/debug/tmp-outdir/b34_7ebw/submol.json",
        "basename": "submol.json",
        "nameroot": "submol",
        "nameext": ".json",
        "class": "File",
        "checksum": "sha1$884f2c965978f27fdd67960b08a33bdd9fd25b16",
        "size": 1206,
        "http://commonwl.org/cwltool#generation": 0
    },
    "input_fasta": {
        "class": "File",
        "location": "file:///pgap/user_input/ASM2732v1.annotation.nucleotide.1.fasta",
        "size": 588482,
        "basename": "ASM2732v1.annotation.nucleotide.1.fasta",
        "nameroot": "ASM2732v1.annotation.nucleotide.1",
        "nameext": ".fasta"
    },
    "no_internet": null,
    "taxon_db": {
        "class": "File",
        "location": "file:///pgap/input/uniColl_path/taxonomy.sqlite3",
        "basename": "taxonomy.sqlite3",
        "size": 1072103424,
        "nameroot": "taxonomy",
        "nameext": ".sqlite3"
    },
    "ifmt": "JSON",
    "ofmt": "JSON",
    "output_annotation_name": "input.asn",
    "output_input_asn_type_name": "input_asn_type.txt",
    "output_ltp_name": "genome.ltp.txt",
    "output_taxid_name": "taxid.txt"
}
[2022-11-28 22:41:56] DEBUG [job pgapx_yaml_ctl] path mappings is {
    "file:///pgap/output/debug/tmp-outdir/b34_7ebw/submol.json": [
        "/pgap/output/debug/tmp-outdir/b34_7ebw/submol.json",
        "/pgap/output/debug/tmpdir/www97fqe/stg8e984d73-30da-4e44-a166-48efa45f0906/submol.json",
        "File",
        true
    ],
    "file:///pgap/user_input/ASM2732v1.annotation.nucleotide.1.fasta": [
        "/pgap/user_input/ASM2732v1.annotation.nucleotide.1.fasta",
        "/pgap/output/debug/tmpdir/www97fqe/stg6f426774-f19e-4fdf-93f5-d59ec24ef4a2/ASM2732v1.annotation.nucleotide.1.fasta",
        "File",
        true
    ],
    "file:///pgap/input/uniColl_path/taxonomy.sqlite3": [
        "/pgap/input/uniColl_path/taxonomy.sqlite3",
        "/pgap/output/debug/tmpdir/www97fqe/stge9037688-e893-4cae-b17a-2be9d8174b1c/taxonomy.sqlite3",
        "File",
        true
    ]
}
[2022-11-28 22:41:56] DEBUG [job pgapx_yaml_ctl] command line bindings is [
    {
        "position": [
            -1000000,
            0
        ],
        "datum": "pgapx_yaml_ctl"
    },
    {
        "prefix": "-ifmt",
        "position": [
            0,
            "ifmt"
        ],
        "datum": "JSON"
    },
    {
        "prefix": "-input",
        "position": [
            0,
            "input"
        ],
        "datum": {
            "location": "file:///pgap/output/debug/tmp-outdir/b34_7ebw/submol.json",
            "basename": "submol.json",
            "nameroot": "submol",
            "nameext": ".json",
            "class": "File",
            "checksum": "sha1$884f2c965978f27fdd67960b08a33bdd9fd25b16",
            "size": 1206,
            "http://commonwl.org/cwltool#generation": 0,
            "path": "/pgap/output/debug/tmpdir/www97fqe/stg8e984d73-30da-4e44-a166-48efa45f0906/submol.json",
            "dirname": "/pgap/output/debug/tmpdir/www97fqe/stg8e984d73-30da-4e44-a166-48efa45f0906"
        }
    },
    {
        "prefix": "-input-fasta",
        "position": [
            0,
            "input_fasta"
        ],
        "datum": {
            "class": "File",
            "location": "file:///pgap/user_input/ASM2732v1.annotation.nucleotide.1.fasta",
            "size": 588482,
            "basename": "ASM2732v1.annotation.nucleotide.1.fasta",
            "nameroot": "ASM2732v1.annotation.nucleotide.1",
            "nameext": ".fasta",
            "path": "/pgap/output/debug/tmpdir/www97fqe/stg6f426774-f19e-4fdf-93f5-d59ec24ef4a2/ASM2732v1.annotation.nucleotide.1.fasta",
            "dirname": "/pgap/output/debug/tmpdir/www97fqe/stg6f426774-f19e-4fdf-93f5-d59ec24ef4a2"
        }
    },
    {
        "prefix": "-ofmt",
        "position": [
            0,
            "ofmt"
        ],
        "datum": "JSON"
    },
    {
        "prefix": "-output-annotation",
        "position": [
            0,
            "output_annotation_name"
        ],
        "datum": "input.asn"
    },
    {
        "prefix": "-output-asn-type",
        "position": [
            0,
            "output_input_asn_type_name"
        ],
        "datum": "input_asn_type.txt"
    },
    {
        "prefix": "-output-ltp",
        "position": [
            0,
            "output_ltp_name"
        ],
        "datum": "genome.ltp.txt"
    },
    {
        "prefix": "-output-taxid",
        "position": [
            0,
            "output_taxid_name"
        ],
        "datum": "taxid.txt"
    },
    {
        "prefix": "-taxon-db",
        "position": [
            0,
            "taxon_db"
        ],
        "datum": {
            "class": "File",
            "location": "file:///pgap/input/uniColl_path/taxonomy.sqlite3",
            "basename": "taxonomy.sqlite3",
            "size": 1072103424,
            "nameroot": "taxonomy",
            "nameext": ".sqlite3",
            "path": "/pgap/output/debug/tmpdir/www97fqe/stge9037688-e893-4cae-b17a-2be9d8174b1c/taxonomy.sqlite3",
            "dirname": "/pgap/output/debug/tmpdir/www97fqe/stge9037688-e893-4cae-b17a-2be9d8174b1c"
        }
    }
]
[2022-11-28 22:41:56] DEBUG [job pgapx_yaml_ctl] initial work dir {}
[2022-11-28 22:41:56] INFO [job pgapx_yaml_ctl] /pgap/output/debug/tmp-outdir/ycchyugf$ pgapx_yaml_ctl \
    -ifmt \
    JSON \
    -input \
    /pgap/output/debug/tmpdir/www97fqe/stg8e984d73-30da-4e44-a166-48efa45f0906/submol.json \
    -input-fasta \
    /pgap/output/debug/tmpdir/www97fqe/stg6f426774-f19e-4fdf-93f5-d59ec24ef4a2/ASM2732v1.annotation.nucleotide.1.fasta \
    -ofmt \
    JSON \
    -output-annotation \
    input.asn \
    -output-asn-type \
    input_asn_type.txt \
    -output-ltp \
    genome.ltp.txt \
    -output-taxid \
    taxid.txt \
    -taxon-db \
    /pgap/output/debug/tmpdir/www97fqe/stge9037688-e893-4cae-b17a-2be9d8174b1c/taxonomy.sqlite3
ignoring failure of pgapx_yaml_input.GetFasta
DEBUG: circular: is_complete = 1
DEBUG: GetOrgRef returned: Org-ref ::= {
  taxname "Mycoplasma genitalium",
  db {
    {
      db "taxon",
      tag id 2097
    }
  },
  orgname {
    name binomial {
      genus "Mycoplasma",
      species "genitalium"
    },
    attrib "specified",
    lineage "Bacteria; Tenericutes; Mollicutes; Mycoplasmataceae; Mycoplasma",
    gcode 4,
    div "BCT"
  }
}
CMacroFunction_LookupPub::s_GetArticleFromEntrezById(): error fetching ID 16397293:  Frame type= eFrameNamed, Object type= Pubmed-entry
DEBUG: molinfo_found=true
DEBUG: CSeqdesc::e_Molinfo: Seqdesc ::= molinfo {
  biomol genomic,
  completeness complete
}
DEBUG: source_found=true
Error: (802.5) CObjectOStream: error at line 126: Seq-entry.seq.descr.E.pub.pub.E: cannot write empty choice
terminate called after throwing an instance of 'ncbi::CSerialException'
  what():  NCBI C++ Exception:
    Error: SERIAL(CSerialException::eInvalidData) "/export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/src/serial/choice.cpp", line 449: ncbi::CChoiceTypeInfoFunctions::WriteChoiceDefault() --- line 126: cannot write empty choice ( at Seq-entry.seq.descr.[].[].pub.pub.[].[])
     Stack trace:
      /panfs/pan1.be-md.ncbi.nlm.nih.gov/gpipe/bacterial_pipeline/system/2022-10-03.build6384/arch/x86_64/lib/libtaxon1.so /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/include/serial/exception.hpp:67 ncbi::CSerialException::CSerialException(ncbi::CDiagCompileInfo const&, ncbi::CException const*, ncbi::CSerialException::EErrCode, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, ncbi::EDiagSev) offset=0x0 addr=0x7fa0f0a0f115
      /panfs/pan1.be-md.ncbi.nlm.nih.gov/gpipe/bacterial_pipeline/system/2022-10-03.build6384/arch/x86_64/lib/libxser.so /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/src/serial/objostr.cpp:547 ncbi::CObjectOStream::ThrowError1(ncbi::CDiagCompileInfo const&, int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, ncbi::CException*) offset=0x0 addr=0x7fa0e818c011
      /panfs/pan1.be-md.ncbi.nlm.nih.gov/gpipe/bacterial_pipeline/system/2022-10-03.build6384/arch/x86_64/lib/libxser.so /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/src/serial/objostr.cpp:510 ncbi::CObjectOStream::ThrowError1(ncbi::CDiagCompileInfo const&, int, char const*, ncbi::CException*) offset=0x0 addr=0x7fa0e818c29f
      /panfs/pan1.be-md.ncbi.nlm.nih.gov/gpipe/bacterial_pipeline/system/2022-10-03.build6384/arch/x86_64/lib/libxser.so /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/src/serial/choice.cpp:449 ncbi::CChoiceTypeInfoFunctions::WriteChoiceDefault(ncbi::CObjectOStream&, ncbi::CTypeInfo const*, void const*) offset=0x0 addr=0x7fa0e8167060
       /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/include/serial/impl/objostr.inl:109 WriteData offset=0x0 addr=(nil)
       /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/src/serial/objostrasn.cpp:606 WriteObject offset=0x0 addr=(nil)
      /panfs/pan1.be-md.ncbi.nlm.nih.gov/gpipe/bacterial_pipeline/system/2022-10-03.build6384/arch/x86_64/lib/libxser.so /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/include/serial/impl/typeinfo.inl:75 ncbi::CObjectOStreamAsn::WriteContainer(ncbi::CContainerTypeInfo const*, void const*) offset=0x0 addr=0x7fa0e81a66c7
      /panfs/pan1.be-md.ncbi.nlm.nih.gov/gpipe/bacterial_pipeline/system/2022-10-03.build6384/arch/x86_64/lib/libxser.so /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/src/serial/classinfo.cpp:373 ncbi::CClassTypeInfo::WriteImplicitMember(ncbi::CObjectOStream&, ncbi::CTypeInfo const*, void const*) offset=0x0 addr=0x7fa0e8162a8a
       /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/include/serial/impl/objostr.inl:109 WriteData offset=0x0 addr=(nil)
       /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/src/serial/objostrasn.cpp:711 WriteObject offset=0x0 addr=(nil)
      /panfs/pan1.be-md.ncbi.nlm.nih.gov/gpipe/bacterial_pipeline/system/2022-10-03.build6384/arch/x86_64/lib/libxser.so /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/include/serial/impl/typeinfo.inl:75 ncbi::CObjectOStreamAsn::WriteClassMember(ncbi::CMemberId const&, ncbi::CTypeInfo const*, void const*) offset=0x0 addr=0x7fa0e81a5b66
       /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/src/serial/objostrasn.cpp:694 WriteMember offset=0x0 addr=(nil)
      /panfs/pan1.be-md.ncbi.nlm.nih.gov/gpipe/bacterial_pipeline/system/2022-10-03.build6384/arch/x86_64/lib/libxser.so /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/include/serial/impl/member.inl:118 ncbi::CObjectOStreamAsn::WriteClass(ncbi::CClassTypeInfo const*, void const*) offset=0x0 addr=0x7fa0e81a4323
       /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/src/serial/choice.cpp:456 WriteVariant offset=0x0 addr=(nil)
      /panfs/pan1.be-md.ncbi.nlm.nih.gov/gpipe/bacterial_pipeline/system/2022-10-03.build6384/arch/x86_64/lib/libxser.so /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/include/serial/impl/variant.inl:172 ncbi::CChoiceTypeInfoFunctions::WriteChoiceDefault(ncbi::CObjectOStream&, ncbi::CTypeInfo const*, void const*) offset=0x0 addr=0x7fa0e81670d2
       /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/include/serial/impl/objostr.inl:109 WriteData offset=0x0 addr=(nil)
       /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/src/serial/objostrasn.cpp:606 WriteObject offset=0x0 addr=(nil)
      /panfs/pan1.be-md.ncbi.nlm.nih.gov/gpipe/bacterial_pipeline/system/2022-10-03.build6384/arch/x86_64/lib/libxser.so /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/include/serial/impl/typeinfo.inl:75 ncbi::CObjectOStreamAsn::WriteContainer(ncbi::CContainerTypeInfo const*, void const*) offset=0x0 addr=0x7fa0e81a66c7
      /panfs/pan1.be-md.ncbi.nlm.nih.gov/gpipe/bacterial_pipeline/system/2022-10-03.build6384/arch/x86_64/lib/libxser.so /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/src/serial/classinfo.cpp:373 ncbi::CClassTypeInfo::WriteImplicitMember(ncbi::CObjectOStream&, ncbi::CTypeInfo const*, void const*) offset=0x0 addr=0x7fa0e8162a8a
      /panfs/pan1.be-md.ncbi.nlm.nih.gov/gpipe/bacterial_pipeline/system/2022-10-03.build6384/arch/x86_64/lib/libxser.so /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/src/serial/typeinfo.cpp:527 ncbi::CTypeInfoFunctions::WriteWithHook(ncbi::CObjectOStream&, ncbi::CTypeInfo const*, void const*) offset=0x0 addr=0x7fa0e8122077
       /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/include/serial/impl/objostr.inl:109 WriteData offset=0x0 addr=(nil)
       /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/src/serial/objostrasn.cpp:711 WriteObject offset=0x0 addr=(nil)
      /panfs/pan1.be-md.ncbi.nlm.nih.gov/gpipe/bacterial_pipeline/system/2022-10-03.build6384/arch/x86_64/lib/libxser.so /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/include/serial/impl/typeinfo.inl:75 ncbi::CObjectOStreamAsn::WriteClassMember(ncbi::CMemberId const&, ncbi::CTypeInfo const*, void const*) offset=0x0 addr=0x7fa0e81a5b66
       /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/src/serial/objostrasn.cpp:694 WriteMember offset=0x0 addr=(nil)
      /panfs/pan1.be-md.ncbi.nlm.nih.gov/gpipe/bacterial_pipeline/system/2022-10-03.build6384/arch/x86_64/lib/libxser.so /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/include/serial/impl/member.inl:118 ncbi::CObjectOStreamAsn::WriteClass(ncbi::CClassTypeInfo const*, void const*) offset=0x0 addr=0x7fa0e81a4323
       /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/src/serial/choice.cpp:456 WriteVariant offset=0x0 addr=(nil)
      /panfs/pan1.be-md.ncbi.nlm.nih.gov/gpipe/bacterial_pipeline/system/2022-10-03.build6384/arch/x86_64/lib/libxser.so /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/include/serial/impl/variant.inl:172 ncbi::CChoiceTypeInfoFunctions::WriteChoiceDefault(ncbi::CObjectOStream&, ncbi::CTypeInfo const*, void const*) offset=0x0 addr=0x7fa0e81670d2
       /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/include/serial/impl/objostr.inl:109 WriteData offset=0x0 addr=(nil)
       /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/src/serial/objostr.cpp:608 WriteObject offset=0x0 addr=(nil)
      /panfs/pan1.be-md.ncbi.nlm.nih.gov/gpipe/bacterial_pipeline/system/2022-10-03.build6384/arch/x86_64/lib/libxser.so /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/include/serial/impl/typeinfo.inl:75 ncbi::CObjectOStream::Write(void const*, ncbi::CTypeInfo const*) offset=0x0 addr=0x7fa0e818acc3
      /panfs/pan1.be-md.ncbi.nlm.nih.gov/gpipe/bacterial_pipeline/system/2022-10-03.build6384/arch/x86_64/lib/libxser.so /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/src/serial/serialobject.cpp:826 ncbi::WriteObject(std::ostream&, void const*, ncbi::CTypeInfo const*) offset=0x0 addr=0x7fa0e81f0348
      /panfs/pan1.be-md.ncbi.nlm.nih.gov/gpipe/bacterial_pipeline/system/2022-10-03.build6384/arch/x86_64/bin/pgapx_yaml_ctl :0  offset=0x0 addr=0x41a117
      /panfs/pan1.be-md.ncbi.nlm.nih.gov/gpipe/bacterial_pipeline/system/2022-10-03.build6384/arch/x86_64/lib/libxncbi.so /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/src/corelib/ncbiapp.cpp:701 ncbi::CNcbiApplicationAPI::x_TryMain(ncbi::EAppDiagStream, char const*, int*, bool*) offset=0x0 addr=0x7fa0e772de72
      /panfs/pan1.be-md.ncbi.nlm.nih.gov/gpipe/bacterial_pipeline/system/2022-10-03.build6384/arch/x86_64/lib/libxncbi.so /export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/src/corelib/ncbiapp.cpp:1001 ncbi::CNcbiApplicationAPI::AppMain(int, char const* const*, char const* const*, ncbi::EAppDiagStream, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) offset=0x0 addr=0x7fa0e77314a4
      /panfs/pan1.be-md.ncbi.nlm.nih.gov/gpipe/bacterial_pipeline/system/2022-10-03.build6384/arch/x86_64/bin/pgapx_yaml_ctl :0  offset=0x0 addr=0x40d5c9
      /usr/lib64/libc-2.17.so :0  offset=0x0 addr=0x7fa0e5f41554
      /panfs/pan1.be-md.ncbi.nlm.nih.gov/gpipe/bacterial_pipeline/system/2022-10-03.build6384/arch/x86_64/bin/pgapx_yaml_ctl :0  offset=0x0 addr=0x40d799
       :0  offset=0x0 addr=0xffffffffffffffff

Stack trace (most recent call last):
#14   Object "", at 0xffffffffffffffff, in 
#13   Object "/panfs/pan1.be-md.ncbi.nlm.nih.gov/gpipe/bacterial_pipeline/system/2022-10-03.build6384/arch/x86_64/bin/pgapx_yaml_ctl", at 0x40d799, in _start
#12   Object "/usr/lib64/libc-2.17.so", at 0x7fa0e5f41554, in __libc_start_main
#11   Object "/panfs/pan1.be-md.ncbi.nlm.nih.gov/gpipe/bacterial_pipeline/system/2022-10-03.build6384/arch/x86_64/bin/pgapx_yaml_ctl", at 0x40d5c9, in main
#10   Source "/export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/src/corelib/ncbiapp.cpp", line 1001, in AppMain [0x7fa0e77314a4]
#9    Source "/export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/src/corelib/ncbiapp.cpp", line 701, in x_TryMain [0x7fa0e772de72]
#8    Object "/panfs/pan1.be-md.ncbi.nlm.nih.gov/gpipe/bacterial_pipeline/system/2022-10-03.build6384/arch/x86_64/bin/pgapx_yaml_ctl", at 0x41a117, in CPgapxYamlCtlApplication::Run()
#7    Source "/export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/src/serial/serialobject.cpp", line 826, in WriteObject [0x7fa0e81f0348]
#6    Source "/export/home/gpipe/TeamCity/Agent3/work/427aceaa834ecbb6/ncbi_cxx/src/serial/objostr.cpp", line 612, in Write [0x7fa0e818ae7a]
#5    Source "../../../../gcc-7.3.0/libstdc++-v3/libsupc++/eh_throw.cc", line 131, in __cxa_rethrow [0x7fa0e6ac3275]
#4    Source "../../../../gcc-7.3.0/libstdc++-v3/libsupc++/eh_terminate.cc", line 57, in terminate [0x7fa0e6ac2fe0]
#3    Source "../../../../gcc-7.3.0/libstdc++-v3/libsupc++/eh_terminate.cc", line 47, in __cxa_begin_catch [0x7fa0e6ac2f95]
#2    Source "../../../../gcc-7.3.0/libstdc++-v3/libsupc++/vterminate.cc", line 95, in __verbose_terminate_handler [0x7fa0e6ac51a4]
#1    Object "/usr/lib64/libc-2.17.so", at 0x7fa0e5f56a77, in abort
#0    Object "/usr/lib64/libc-2.17.so", at 0x7fa0e5f55387, in raise
Aborted (Signal sent by tkill() 227 1003)
[2022-11-28 22:42:02] INFO [job pgapx_yaml_ctl] Max memory used: 37MiB
[2022-11-28 22:42:02] WARNING [job pgapx_yaml_ctl] was terminated by signal: SIGABRT
[2022-11-28 22:42:02] ERROR [job pgapx_yaml_ctl] Job error:
("Error collecting output for parameter 'input_asn_type': pgap/progs/pgapx_yaml_ctl.cwl:75:13: Did not find output file with glob pattern: '['input_asn_type.txt']'.", {})
[2022-11-28 22:42:02] WARNING [job pgapx_yaml_ctl] completed permanentFail
[2022-11-28 22:42:02] DEBUG [job pgapx_yaml_ctl] outputs {}
[2022-11-28 22:42:02] ERROR [step pgapx_yaml_ctl] Output is missing expected field file:///pgap/pgap/prepare_user_input2.cwl#pgapx_yaml_ctl/output_annotation
[2022-11-28 22:42:02] ERROR [step pgapx_yaml_ctl] Output is missing expected field file:///pgap/pgap/prepare_user_input2.cwl#pgapx_yaml_ctl/output_ltp
[2022-11-28 22:42:02] ERROR [step pgapx_yaml_ctl] Output is missing expected field file:///pgap/pgap/prepare_user_input2.cwl#pgapx_yaml_ctl/input_asn_type
[2022-11-28 22:42:02] ERROR [step pgapx_yaml_ctl] Output is missing expected field file:///pgap/pgap/prepare_user_input2.cwl#pgapx_yaml_ctl/taxid
[2022-11-28 22:42:02] DEBUG [step pgapx_yaml_ctl] produced output {}
[2022-11-28 22:42:02] WARNING [step pgapx_yaml_ctl] completed permanentFail
[2022-11-28 22:42:02] DEBUG [job pgapx_yaml_ctl] Removing input staging directory /pgap/output/debug/tmpdir/www97fqe
[2022-11-28 22:42:02] INFO [workflow prepare_input_template] completed permanentFail
[2022-11-28 22:42:02] DEBUG [workflow prepare_input_template] outputs {
    "input_asn_type": null,
    "locus_tag_prefix": null,
    "output_entries": null,
    "output_seq_submit": null,
    "submol_block_json": {
        "location": "file:///pgap/output/debug/tmp-outdir/b34_7ebw/submol.json",
        "basename": "submol.json",
        "nameroot": "submol",
        "nameext": ".json",
        "class": "File",
        "checksum": "sha1$884f2c965978f27fdd67960b08a33bdd9fd25b16",
        "size": 1206,
        "http://commonwl.org/cwltool#generation": 0
    },
    "taxid": null
}
[2022-11-28 22:42:02] DEBUG [step prepare_input_template] produced output {
    "file:///pgap/pgap/pgap.cwl#prepare_input_template/output_seq_submit": null,
    "file:///pgap/pgap/pgap.cwl#prepare_input_template/output_entries": null,
    "file:///pgap/pgap/pgap.cwl#prepare_input_template/locus_tag_prefix": null,
    "file:///pgap/pgap/pgap.cwl#prepare_input_template/submol_block_json": {
        "location": "file:///pgap/output/debug/tmp-outdir/b34_7ebw/submol.json",
        "basename": "submol.json",
        "nameroot": "submol",
        "nameext": ".json",
        "class": "File",
        "checksum": "sha1$884f2c965978f27fdd67960b08a33bdd9fd25b16",
        "size": 1206,
        "http://commonwl.org/cwltool#generation": 0
    },
    "file:///pgap/pgap/pgap.cwl#prepare_input_template/taxid": null
}
[2022-11-28 22:42:02] WARNING [step prepare_input_template] completed permanentFail
[2022-11-28 22:42:02] INFO [workflow ] completed permanentFail
[2022-11-28 22:42:02] DEBUG [workflow ] outputs {
    "calls": {
        "location": "file:///pgap/output/debug/tmp-outdir/i193unuf/calls.tab",
        "basename": "calls.tab",
        "nameroot": "calls",
        "nameext": ".tab",
        "class": "File",
        "checksum": "sha1$da39a3ee5e6b4b0d3255bfef95601890afd80709",
        "size": 0,
        "http://commonwl.org/cwltool#generation": 0
    },
    "checkm_raw": null,
    "final_asndisc_error_diag": null,
    "final_asnval_error_diag": null,
    "gbk": null,
    "gff": null,
    "initial_asndisc_error_diag": null,
    "initial_asnval_error_diag": null,
    "input_fasta": {
        "class": "File",
        "location": "file:///pgap/user_input/ASM2732v1.annotation.nucleotide.1.fasta",
        "size": 588482,
        "basename": "ASM2732v1.annotation.nucleotide.1.fasta",
        "nameroot": "ASM2732v1.annotation.nucleotide.1",
        "nameext": ".fasta"
    },
    "input_submol": {
        "class": "File",
        "location": "file:///pgap/user_input/pgap_submol_3kfbirdg.yaml",
        "size": 1667,
        "basename": "pgap_submol_3kfbirdg.yaml",
        "nameroot": "pgap_submol_3kfbirdg",
        "nameext": ".yaml"
    },
    "nucleotide_fasta": null,
    "protein_fasta": null,
    "sqn": null
}
[2022-11-28 22:42:02] DEBUG Copying /pgap/output/debug/tmp-outdir/i193unuf/calls.tab to /pgap/output/calls.tab
[2022-11-28 22:42:02] DEBUG Copying /pgap/user_input/ASM2732v1.annotation.nucleotide.1.fasta to /pgap/output/ASM2732v1.annotation.nucleotide.1.fasta
[2022-11-28 22:42:02] DEBUG Copying /pgap/user_input/pgap_submol_3kfbirdg.yaml to /pgap/output/pgap_submol_3kfbirdg.yaml
[2022-11-28 22:42:02] WARNING Final process status is permanentFail
{
    "calls": {
        "location": "file:///pgap/output/calls.tab",
        "basename": "calls.tab",
        "class": "File",
        "checksum": "sha1$da39a3ee5e6b4b0d3255bfef95601890afd80709",
        "size": 0,
        "path": "/pgap/output/calls.tab"
    },
    "checkm_raw": null,
    "final_asndisc_error_diag": null,
    "final_asnval_error_diag": null,
    "gbk": null,
    "gff": null,
    "initial_asndisc_error_diag": null,
    "initial_asnval_error_diag": null,
    "input_fasta": {
        "class": "File",
        "location": "file:///pgap/output/ASM2732v1.annotation.nucleotide.1.fasta",
        "size": 588482,
        "basename": "ASM2732v1.annotation.nucleotide.1.fasta",
        "checksum": "sha1$f6129783cc8562db7bca3c87310d57d8dd07ce2c",
        "path": "/pgap/output/ASM2732v1.annotation.nucleotide.1.fasta"
    },
    "input_submol": {
        "class": "File",
        "location": "file:///pgap/output/pgap_submol_3kfbirdg.yaml",
        "size": 1667,
        "basename": "pgap_submol_3kfbirdg.yaml",
        "checksum": "sha1$c76170f856551e6f31a13cd29e4ab394c153385f",
        "path": "/pgap/output/pgap_submol_3kfbirdg.yaml"
    },
    "nucleotide_fasta": null,
    "protein_fasta": null,
    "sqn": null
}
azat-badretdin commented 1 year ago

Thank you for your report! This is a known problem related to third party operations and the fix will be provided in the next release, in December.

azat-badretdin commented 1 year ago

The workaround is to avoid using literature references in your input submol.yaml file by removing publications: section.

azat-badretdin commented 1 year ago

The workaround is to avoid using literature references in your input submol.yaml file by removing publications: section

I have added a somewhat simpler workaround option by modifying, temporarily, out test case to eliminate aforementioned publications: section, so you can, if you want, wipe out existing installation and reinstall PGAPx with new test data....

ac-simpson commented 1 year ago

Hi Azat, thanks very much. Unfortunately when I deleted pgap and tried to reinstall (Which I did successfully 5 or 6 times yesterday) the installation is failing every time. I have tried both on our ubuntu server and on my own laptop (Macbook Pro). Specifically, 

ERROR: Failed to extract tarball; to install manually, try something like:
    curl -OLC - https://s3.amazonaws.com/pgap-data/test_genomes-2022-10-03.build6384.tgz
    tar xvf test_genomes-2022-10-03.build6384.tgz

I can't tell if anything else is going wrong.

azat-badretdin commented 1 year ago

Hmm... Looks like AWS told me a lie by telling me that this tarball is Public....

azat-badretdin commented 1 year ago

I got help to fix this in place. Permissions are now for public, feel free to retry.

ac-simpson commented 1 year ago

Success! Well, someone is hogging our whole server so I started the process on my laptop - and it actually started the script instead of erroring out. Thanks!