bigbio / quantms

Quantitative mass spectrometry workflow.
MIT License
24 stars 10 forks source link

Error memory TMT dataset PXD012574 #246

Open ypriverol opened 1 year ago

ypriverol commented 1 year ago

Description of the bug

URL of the working file: http://ftp.pride.ebi.ac.uk/pub/databases/pride/resources/proteomes/proteogenomics/benchmakrs/15acadc82868c799e567dd1b219b44/


nf-core/quantms execution completed unsuccessfully!

The exit status of the task that caused the workflow execution to fail was: 130.

The full error message was:

Error executing process > 'NFCORE_QUANTMS:QUANTMS:TMT:PROTEININFERENCE:PROTEININFERENCER'

Caused by:
  Process `NFCORE_QUANTMS:QUANTMS:TMT:PROTEININFERENCE:PROTEININFERENCER` terminated with an error exit status (130)

Command executed:

  ProteinInference \
      -in ID_mapper_merge.consensusXML \
      -threads 48 \
      -picked_fdr true \
      -picked_decoy_string DECOY_ \
      -protein_fdr true \
      -Algorithm:use_shared_peptides true \
      -Algorithm:annotate_indistinguishable_groups true \
       \
      -Algorithm:score_aggregation_method best \
      -Algorithm:min_peptides_per_protein 1 \
      -out ID_mapper_merge_epi.consensusXML \
      -debug 0 \
      |& tee ID_mapper_merge_inference.log

  cat <<-END_VERSIONS > versions.yml
  "NFCORE_QUANTMS:QUANTMS:TMT:PROTEININFERENCE:PROTEININFERENCER":
      ProteinInference: $(ProteinInference 2>&1 | grep -E '^Version(.*) ' | sed 's/Version: //g' | cut -d ' ' -f 1)
  END_VERSIONS

Command exit status:
  130

Command output:
  Loading input...
  Loading input took 41.49 s (wall), 41.40 s (CPU), 0.63 s (system), 40.77 s (user)
   occurred 2 times
  Merging IDs across runs...
  Merging IDs across runs took 0.00 s (wall), 0.00 s (CPU), 0.00 s (system), 0.00 s (user)
  Aggregating protein scores...
  Building graph on 738370 features, 18751 unassigned spectra (if chosen) and 22415 proteins.
  Progress of 'Building graph...':

  -- done [took 0.66 s (CPU), 0.66 s (Wall)] -- 
  Progress of 'Annotating indistinguishable proteins...':

  -- done [took 1.23 s (CPU), 0.07 s (Wall)] -- 
  Found 17785 connected components.
  Aggregating protein scores took 0.00 s (wall), 0.00 s (CPU), 0.00 s (system), 0.00 s (user)
  Calculating target-decoy q-values...
  Storing output...
  ConsensusXMLFile::store():  found 1 invalid unique ids

Command wrapper:
  Aggregating protein scores...
  Building graph on 738370 features, 18751 unassigned spectra (if chosen) and 22415 proteins.
  Progress of 'Building graph...':

  -- done [took 0.66 s (CPU), 0.66 s (Wall)] -- 
  Progress of 'Annotating indistinguishable proteins...':

  -- done [took 1.23 s (CPU), 0.07 s (Wall)] -- 
  Found 17785 connected components.
  Aggregating protein scores took 0.00 s (wall), 0.00 s (CPU), 0.00 s (system), 0.00 s (user)
  Calculating target-decoy q-values...
  Storing output...
  ConsensusXMLFile::store():  found 1 invalid unique ids

  ------------------------------------------------------------
  Sender: LSF System 
  Subject: Job 7751604:  in cluster  Exited

  Job  was submitted from host  by user  in cluster  at Thu Nov 24 18:00:09 2022
  Job was executed on host(s) <48*hl-codon-bm-07>, in queue , as user  in cluster  at Thu Nov 24 18:00:19 2022
   was used as the home directory.
   was used as the working directory.
  Started at Thu Nov 24 18:00:19 2022
  Terminated at Thu Nov 24 18:10:28 2022
  Results reported at Thu Nov 24 18:10:28 2022

  Your job looked like:

  ------------------------------------------------------------
  # LSBATCH: User input
  ------------------------------------------------------------

  TERM_MEMLIMIT: job killed after reaching LSF memory usage limit.
  Exited with exit code 130.

  Resource usage summary:

      CPU time :                                   612.00 sec.
      Max Memory :                                 318248 MB
      Average Memory :                             151518.11 MB
      Total Requested Memory :                     307200.00 MB
      Delta Memory :                               -11048.00 MB
      Max Swap :                                   -
      Max Processes :                              13
      Max Threads :                                68
      Run time :                                   608 sec.
      Turnaround time :                            619 sec.

  The output (if any) is above this job summary.

Work dir:
  /hps/nobackup/juan/pride/reanalysis/differential-expression/tmt/PXD012574/work/39/15acadc82868c799e567dd1b219b44

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`

Command used and terminal output

No response

Relevant files

No response

System information

No response

ypriverol commented 1 year ago

Even with more memory, it continues failing:

Error executing process > 'NFCORE_QUANTMS:QUANTMS:TMT:PROTEININFERENCE:PROTEININFERENCER'

Caused by:
  Process `NFCORE_QUANTMS:QUANTMS:TMT:PROTEININFERENCE:PROTEININFERENCER` terminated with an error exit status (139)

Command executed:

  ProteinInference \
      -in ID_mapper_merge.consensusXML \
      -threads 48 \
      -picked_fdr true \
      -picked_decoy_string DECOY_ \
      -protein_fdr true \
      -Algorithm:use_shared_peptides true \
      -Algorithm:annotate_indistinguishable_groups true \
       \
      -Algorithm:score_aggregation_method best \
      -Algorithm:min_peptides_per_protein 1 \
      -out ID_mapper_merge_epi.consensusXML \
      -debug 0 \
      |& tee ID_mapper_merge_inference.log

  cat <<-END_VERSIONS > versions.yml
  "NFCORE_QUANTMS:QUANTMS:TMT:PROTEININFERENCE:PROTEININFERENCER":
      ProteinInference: $(ProteinInference 2>&1 | grep -E '^Version(.*) ' | sed 's/Version: //g' | cut -d ' ' -f 1)
  END_VERSIONS

Command exit status:
  139

Command output:
  Loading input...
  Loading input took 41.83 s (wall), 41.62 s (CPU), 0.76 s (system), 40.86 s (user)
  <Loading input...> occurred 2 times
  Merging IDs across runs...
  Merging IDs across runs took 0.00 s (wall), 0.00 s (CPU), 0.00 s (system), 0.00 s (user)
  Aggregating protein scores...
  Building graph on 738370 features, 18751 unassigned spectra (if chosen) and 22415 proteins.
  Progress of 'Building graph...':

    9.92 %               
  -- done [took 0.59 s (CPU), 0.60 s (Wall)] -- 
  Progress of 'Annotating indistinguishable proteins...':

  -- done [took 2.60 s (CPU), 0.11 s (Wall)] -- 
  Found 17785 connected components.
  Aggregating protein scores took 0.00 s (wall), 0.00 s (CPU), 0.00 s (system), 0.00 s (user)
  Calculating target-decoy q-values...
  Storing output...
  ConsensusXMLFile::store():  found 1 invalid unique ids

Command error:
  .command.sh: line 15: 2292302 Segmentation fault      (core dumped) ProteinInference -in ID_mapper_merge.consensusXML -threads 48 -picked_fdr true -picked_decoy_string DECOY_ -protein_fdr true -Algorithm:use_shared_peptides true -Algorithm:annotate_indistinguishable_groups true -Algorithm:score_aggregation_method best -Algorithm:min_peptides_per_protein 1 -out ID_mapper_merge_epi.consensusXML -debug 0 2>&1
       2292303 Done                    | tee ID_mapper_merge_inference.log

Work dir:
  /hps/nobackup/juan/pride/reanalysis/differential-expression/tmt/PXD012574/work/b7/2d34857a791de24ebf9f93e632526e

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`
ypriverol commented 1 year ago

I have run the dataset only with comet and it finished. @jpfeuffer

jpfeuffer commented 1 year ago

Stacktrace:

Toggle me! ``` #74641 0x00007f82c7ea57b5 in boost::spirit::karma::int_inserter<10u, boost::spirit::unused_type, boost::spirit::unused_type>::call, std::allocator > >, mpl_::int_<0>, boost::spirit::unused_type>, double> (sink=..., n=-nan(0x8000000000000), num=@0x7ffc9e6e4220: inf, exp=315) at /usr/include/boost/spirit/home/karma/numeric/detail/numeric_utils.hpp:597 #74642 0x00007f82c7ea57b5 in boost::spirit::karma::int_inserter<10u, boost::spirit::unused_type, boost::spirit::unused_type>::call, std::allocator > >, mpl_::int_<0>, boost::spirit::unused_type>, double> (sink=..., n=inf, num=@0x7ffc9e6e4220: inf, exp=308) at /usr/include/boost/spirit/home/karma/numeric/detail/numeric_utils.hpp:597 #74643 0x00007f82c7ea57b5 in boost::spirit::karma::int_inserter<10u, boost::spirit::unused_type, boost::spirit::unused_type>::call, std::allocator > >, mpl_::int_<0>, boost::spirit::unused_type>, double> (sink=..., n=inf, num=@0x7ffc9e6e4220: inf, exp=301) at /usr/include/boost/spirit/home/karma/numeric/detail/numeric_utils.hpp:597 #74644 0x00007f82c7ea57b5 in boost::spirit::karma::int_inserter<10u, boost::spirit::unused_type, boost::spirit::unused_type>::call, std::allocator > >, mpl_::int_<0>, boost::spirit::unused_type>, double> (sink=..., n=inf, num=@0x7ffc9e6e4220: inf, exp=294) at /usr/include/boost/spirit/home/karma/numeric/detail/numeric_utils.hpp:597 #74645 0x00007f82c7ea57b5 in boost::spirit::karma::int_inserter<10u, boost::spirit::unused_type, boost::spirit::unused_type>::call, std::allocator > >, mpl_::int_<0>, boost::spirit::unused_type>, double> (sink=..., n=inf, num=@0x7ffc9e6e4220: inf, exp=287) at /usr/include/boost/spirit/home/karma/numeric/detail/numeric_utils.hpp:597 #74646 0x00007f82c7ea57b5 in boost::spirit::karma::int_inserter<10u, boost::spirit::unused_type, boost::spirit::unused_type>::call, std::allocator > >, mpl_::int_<0>, boost::spirit::unused_type>, double> (sink=..., n=inf, num=@0x7ffc9e6e4220: inf, exp=280) at /usr/include/boost/spirit/home/karma/numeric/detail/numeric_utils.hpp:597 #74647 0x00007f82c7ea57b5 in boost::spirit::karma::int_inserter<10u, boost::spirit::unused_type, boost::spirit::unused_type>::call, std::allocator > >, mpl_::int_<0>, boost::spirit::unused_type>, double> (sink=..., n=inf, num=@0x7ffc9e6e4220: inf, exp=273) at /usr/include/boost/spirit/home/karma/numeric/detail/numeric_utils.hpp:597 #74648 0x00007f82c7ea57b5 in boost::spirit::karma::int_inserter<10u, boost::spirit::unused_type, boost::spirit::unused_type>::call, std::allocator > >, mpl_::int_<0>, boost::spirit::unused_type>, double> (sink=..., n=inf, num=@0x7ffc9e6e4220: inf, exp=266) at /usr/include/boost/spirit/home/karma/numeric/detail/numeric_utils.hpp:597 #74649 0x00007f82c7ea57b5 in boost::spirit::karma::int_inserter<10u, boost::spirit::unused_type, boost::spirit::unused_type>::call, std::allocator > >, mpl_::int_<0>, boost::spirit::unused_type>, double> (sink=..., n=inf, num=@0x7ffc9e6e4220: inf, exp=259) at /usr/include/boost/spirit/home/karma/numeric/detail/numeric_utils.hpp:597 #74650 0x00007f82c7ea57b5 in boost::spirit::karma::int_inserter<10u, boost::spirit::unused_type, boost::spirit::unused_type>::call, std::allocator > >, mpl_::int_<0>, boost::spirit::unused_type>, double> (sink=..., n=inf, num=@0x7ffc9e6e4220: inf, exp=252) at /usr/include/boost/spirit/home/karma/numeric/detail/numeric_utils.hpp:597 #74651 0x00007f82c7ea57b5 in boost::spirit::karma::int_inserter<10u, boost::spirit::unused_type, boost::spirit::unused_type>::call, std::allocator > >, mpl_::int_<0>, boost::spirit::unused_type>, double> (sink=..., n=inf, num=@0x7ffc9e6e4220: inf, exp=245) at /usr/include/boost/spirit/home/karma/numeric/detail/numeric_utils.hpp:597 #74652 0x00007f82c7ea57b5 in boost::spirit::karma::int_inserter<10u, boost::spirit::unused_type, boost::spirit::unused_type>::call, std::allocator > >, mpl_::int_<0>, boost::spirit::unused_type>, double> (sink=..., n=inf, num=@0x7ffc9e6e4220: inf, exp=238) at /usr/include/boost/spirit/home/karma/numeric/detail/numeric_utils.hpp:597 ... #74684 0x00007f82c7ea57b5 in boost::spirit::karma::int_inserter<10u, boost::spirit::unused_type, boost::spirit::unused_type>::call, std::allocator > >, mpl_::int_<0>, boost::spirit::unused_type>, double> (sink=..., n=inf, num=@0x7ffc9e6e4220: inf, exp=14) at /usr/include/boost/spirit/home/karma/numeric/detail/numeric_utils.hpp:597 #74685 0x00007f82c7ea57b5 in boost::spirit::karma::int_inserter<10u, boost::spirit::unused_type, boost::spirit::unused_type>::call, std::allocator > >, mpl_::int_<0>, boost::spirit::unused_type>, double> (sink=..., n=inf, num=@0x7ffc9e6e4220: inf, exp=7) at /usr/include/boost/spirit/home/karma/numeric/detail/numeric_utils.hpp:597 #74686 0x00007f82c7ea3d90 in boost::spirit::karma::int_inserter<10u, boost::spirit::unused_type, boost::spirit::unused_type>::call, std::allocator > >, mpl_::int_<0>, boost::spirit::unused_type> > (sink=..., n=inf) at /usr/include/boost/spirit/home/karma/numeric/detail/numeric_utils.hpp:656 #74687 0x00007f82c7ea26a1 in boost::spirit::karma::real_policies::integer_part, std::allocator > >, mpl_::int_<0>, boost::spirit::unused_type> > (sink=..., n=inf, sign=false, force_sign=false) at /usr/include/boost/spirit/home/karma/numeric/real_policies.hpp:192 #74688 0x00007f82c7ea0643 in boost::spirit::karma::real_inserter, boost::spirit::unused_type, boost::spirit::unused_type>::call_n, std::allocator > >, mpl_::int_<0>, boost::spirit::unused_type>, double> (sink=..., n=inf, p=...) at /usr/include/boost/spirit/home/karma/numeric/detail/real_utils.hpp:164 #74689 0x00007f82c7e9e58b in boost::spirit::karma::real_policies::call, boost::spirit::unused_type, boost::spirit::unused_type>, boost::spirit::karma::detail::output_iterator, std::allocator > >, mpl_::int_<0>, boost::spirit::unused_type>, OpenMS::StringConversions::BK_PrecPolicy > (sink=..., n=4.6467477951396374e-310, p=...) at /usr/include/boost/spirit/home/karma/numeric/real_policies.hpp:94 #74690 0x00007f82c7e9d880 in boost::spirit::karma::real_inserter, boost::spirit::unused_type, boost::spirit::unused_type>::call, std::allocator > >, mpl_::int_<0>, boost::spirit::unused_type>, double> (sink=..., n=4.6467477951396374e-310, p=...) at /usr/include/boost/spirit/home/karma/numeric/detail/real_utils.hpp:56 #74691 0x00007f82c7e9c8af in boost::spirit::karma::any_real_generator, boost::spirit::unused_type, boost::spirit::unused_type>::generate, std::allocator > >, mpl_::int_<0>, boost::spirit::unused_type>, boost::spirit::context, boost::spirit::locals >, boost::spirit::unused_type, double> (this=0x7ffc9e6e440f, sink=..., context=..., d=..., attr=@0x7ffc9e6e4488: 4.6467477951396374e-310) at /usr/include/boost/spirit/home/karma/numeric/real.hpp:192 #74692 0x00007f82c7e9bdf8 in boost::spirit::karma::generate, std::allocator > >, mpl_::int_<0>, boost::spirit::karma::real_generator >, double> (sink=..., expr=..., attr=@0x7ffc9e6e4488: 4.6467477951396374e-310) at /usr/include/boost/spirit/home/karma/generate.hpp:69 #74693 0x00007f82c7e9b1c0 in boost::spirit::karma::generate, std::allocator > >, boost::spirit::karma::real_generator >, double> (sink_=..., expr=..., attr=@0x7ffc9e6e4488: 4.6467477951396374e-310) at /usr/include/boost/spirit/home/karma/generate.hpp:91 #74694 0x00007f82c7e9a540 in OpenMS::StringConversions::append (d=4.6467477951396374e-310, target=...) at /workspace/OpenMS/src/openms/include/OpenMS/DATASTRUCTURES/StringConversions.h:205 #74695 0x00007f82c7e98436 in OpenMS::String::String (this=0x7ffc9e6e45e0, d=4.6467477951396374e-310, full_precision=true) at /workspace/OpenMS/src/openms/source/DATASTRUCTURES/String.cpp:147 ``` and then an infinite loop.
jpfeuffer commented 1 year ago

@timosachsenberg

jpfeuffer commented 1 year ago

see https://github.com/OpenMS/OpenMS/issues/6507