BrokenProcessPool

[sarag723]

['F:\Sara\fragpipe\jre\bin\java.exe', '-jar', '-Dfile.encoding=UTF-8', '-Xmx20G', WindowsPath('F:/Sara/fragpipe/tools/MSFragger-4.1/MSFragger-4.1.jar'), '--generate_expect_functions', 'DQB2-notrypsin-DDA_scores_histogram.tsv'] Traceback (most recent call last): File "F:\Sara\fragpipe\tools\msfragger_pep_split.py", line 631, in main() File "F:\Sara\fragpipe\tools\msfragger_pep_split.py", line 623, in main combine_results() File "F:\Sara\fragpipe\tools\msfragger_pep_split.py", line 477, in combine_results e.result() File "F:\Sara\fragpipe\lib\concurrent\futures_base.py", line 439, in result return self.get_result() File "F:\Sara\fragpipe\lib\concurrent\futures_base.py", line 391, in get_result raise self._exception concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending. Process 'MSFragger' finished, exit code: 1 Process returned non-zero exit code, stopping

Cancelling 15 remaining tasks
[log_2024-08-05_12-07-17.txt](https://github.com/user-attachments/files/16532043/log_2024-08-05_12-07-17.txt)

(I am using the non-specific HLA workflow for DDA samples, running them against the uniprot human FASTA database, I used the 'split database' option to split it into 8 and even 10 , but it still crashes )

[fcyu]

Could you share the whole log file?

Thanks,

Fengchao

[sarag723]

log_2024-08-05_12-07-17.txt Here you go, Thanks !

[fcyu]

I think there is a bug in split database script. Guo Ci @guoci , can you take a look? When the mass calibration fails due to not enough high quality, MSFragger uses the original mzML files. But it seems that the split database didn't consider this situation:

********************MASS CALIBRATION AND PARAMETER OPTIMIZATION*******************
-----|---------------|---------------|---------------|---------------
     |  MS1   (Old)  |  MS1   (New)  |  MS2   (Old)  |  MS2   (New)  
-----|---------------|---------------|---------------|---------------
 Run |  Median  MAD  |  Median  MAD  |  Median  MAD  |  Median  MAD  
 001 | Not enough data to perform mass calibration. Using the uncalibrated data.
-----|---------------|---------------|---------------|---------------
Finding the optimal parameters:
-------|-------|-------|-------|-------|-------|-------
  MS2  |    7  |   10  |   15  |   20  |   25  |   30  
-------|-------|-------|-------|-------|-------|-------
 Count |      5|      5|      5|      6|      6|      6
-------|-------|-------|-------|-------|-------|-------
-------|-------|-------|-------|-------
 Peaks | 300_0 | 200_0 | 150_1 | 100_1 
-------|-------|-------|-------|-------
 Count |      6|      6|      6|      6
-------|-------|-------|-------|-------
-------|-------
 Int.  |    0  
-------|-------
 Count |     11
-------|-------
-------|-------
 Rm P. |    0  
-------|-------
 Count |      8
-------|-------
New fragment_mass_tolerance = 20.000000 PPM
New use_topN_peaks = 100
New minimum_ratio = 0.010000
New intensity_transform = 0
New remove_precursor_peak = 1
************MASS CALIBRATION AND PARAMETER OPTIMIZATION DONE IN 13.439 MIN*********
*******************************TOTAL TIME 17.052 MIN********************************
STARTED: DB split 1 of 8
['F:\\Sara\\fragpipe\\jre\\bin\\java.exe', '-jar', '-Dfile.encoding=UTF-8', '-Xmx20G', WindowsPath('F:/Sara/fragpipe/tools/MSFragger-4.1/MSFragger-4.1.jar'), 'fragger.params', WindowsPath('F:/Sara/LCL-project/hla-dqb2-trypsin-notrypsin-080224/dqb2-notrypsin-dda/humanfasta/split_peptide_index_tempdir/DQB2-notrypsin-DDA.mzML'), '--partial', '0']
DONE: DB split 1 of 8
STARTED: DB split 2 of 8
['F:\\Sara\\fragpipe\\jre\\bin\\java.exe', '-jar', '-Dfile.encoding=UTF-8', '-Xmx20G', WindowsPath('F:/Sara/fragpipe/tools/MSFragger-4.1/MSFragger-4.1.jar'), 'fragger.params', WindowsPath('F:/Sara/LCL-project/hla-dqb2-trypsin-notrypsin-080224/dqb2-notrypsin-dda/humanfasta/split_peptide_index_tempdir/DQB2-notrypsin-DDA.mzML'), '--partial', '1']
DONE: DB split 2 of 8
STARTED: DB split 3 of 8
['F:\\Sara\\fragpipe\\jre\\bin\\java.exe', '-jar', '-Dfile.encoding=UTF-8', '-Xmx20G', WindowsPath('F:/Sara/fragpipe/tools/MSFragger-4.1/MSFragger-4.1.jar'), 'fragger.params', WindowsPath('F:/Sara/LCL-project/hla-dqb2-trypsin-notrypsin-080224/dqb2-notrypsin-dda/humanfasta/split_peptide_index_tempdir/DQB2-notrypsin-DDA.mzML'), '--partial', '2']
DONE: DB split 3 of 8
STARTED: DB split 4 of 8
['F:\\Sara\\fragpipe\\jre\\bin\\java.exe', '-jar', '-Dfile.encoding=UTF-8', '-Xmx20G', WindowsPath('F:/Sara/fragpipe/tools/MSFragger-4.1/MSFragger-4.1.jar'), 'fragger.params', WindowsPath('F:/Sara/LCL-project/hla-dqb2-trypsin-notrypsin-080224/dqb2-notrypsin-dda/humanfasta/split_peptide_index_tempdir/DQB2-notrypsin-DDA.mzML'), '--partial', '3']
DONE: DB split 4 of 8
STARTED: DB split 5 of 8
['F:\\Sara\\fragpipe\\jre\\bin\\java.exe', '-jar', '-Dfile.encoding=UTF-8', '-Xmx20G', WindowsPath('F:/Sara/fragpipe/tools/MSFragger-4.1/MSFragger-4.1.jar'), 'fragger.params', WindowsPath('F:/Sara/LCL-project/hla-dqb2-trypsin-notrypsin-080224/dqb2-notrypsin-dda/humanfasta/split_peptide_index_tempdir/DQB2-notrypsin-DDA.mzML'), '--partial', '4']
DONE: DB split 5 of 8
STARTED: DB split 6 of 8
['F:\\Sara\\fragpipe\\jre\\bin\\java.exe', '-jar', '-Dfile.encoding=UTF-8', '-Xmx20G', WindowsPath('F:/Sara/fragpipe/tools/MSFragger-4.1/MSFragger-4.1.jar'), 'fragger.params', WindowsPath('F:/Sara/LCL-project/hla-dqb2-trypsin-notrypsin-080224/dqb2-notrypsin-dda/humanfasta/split_peptide_index_tempdir/DQB2-notrypsin-DDA.mzML'), '--partial', '5']
DONE: DB split 6 of 8
STARTED: DB split 7 of 8
['F:\\Sara\\fragpipe\\jre\\bin\\java.exe', '-jar', '-Dfile.encoding=UTF-8', '-Xmx20G', WindowsPath('F:/Sara/fragpipe/tools/MSFragger-4.1/MSFragger-4.1.jar'), 'fragger.params', WindowsPath('F:/Sara/LCL-project/hla-dqb2-trypsin-notrypsin-080224/dqb2-notrypsin-dda/humanfasta/split_peptide_index_tempdir/DQB2-notrypsin-DDA.mzML'), '--partial', '6']
DONE: DB split 7 of 8
STARTED: DB split 8 of 8
['F:\\Sara\\fragpipe\\jre\\bin\\java.exe', '-jar', '-Dfile.encoding=UTF-8', '-Xmx20G', WindowsPath('F:/Sara/fragpipe/tools/MSFragger-4.1/MSFragger-4.1.jar'), 'fragger.params', WindowsPath('F:/Sara/LCL-project/hla-dqb2-trypsin-notrypsin-080224/dqb2-notrypsin-dda/humanfasta/split_peptide_index_tempdir/DQB2-notrypsin-DDA.mzML'), '--partial', '7']
DONE: DB split 8 of 8
['F:\\Sara\\fragpipe\\jre\\bin\\java.exe', '-jar', '-Dfile.encoding=UTF-8', '-Xmx20G', WindowsPath('F:/Sara/fragpipe/tools/MSFragger-4.1/MSFragger-4.1.jar'), '--generate_expect_functions', 'DQB2-notrypsin-DDA_scores_histogram.tsv']
Traceback (most recent call last):
  File "F:\Sara\fragpipe\tools\msfragger_pep_split.py", line 631, in <module>
    main()
  File "F:\Sara\fragpipe\tools\msfragger_pep_split.py", line 623, in main
    combine_results()
  File "F:\Sara\fragpipe\tools\msfragger_pep_split.py", line 477, in combine_results
    e.result()
  File "F:\Sara\fragpipe\lib\concurrent\futures\_base.py", line 439, in result
    return self.__get_result()
  File "F:\Sara\fragpipe\lib\concurrent\futures\_base.py", line 391, in __get_result
    raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
Process 'MSFragger' finished, exit code: 1
Process returned non-zero exit code, stopping

@sarag723 , for now, could you change "calibration and optimization" to "none"?

Thanks,

Fengchao

[sarag723]

log_2024-08-07_13-24-03.txt I tried changing it to 'none' and it still crashed

[fcyu]

Then, I have no idea. Something was wrong with the database split script because MSFragger was not running. Have to wait for Guo Ci @guoci 's help.

Best,

Fengchao