Nesvilab / philosopher

PeptideProphet, PTMProphet, ProteinProphet, iProphet, Abacus, and FDR filtering
https://philosopher.nesvilab.org
GNU General Public License v3.0
111 stars 19 forks source link

Pipeline stops with error for Integrated Report step (Abacus). #296

Closed fstein closed 2 years ago

fstein commented 2 years ago

Hello,

when running the pipeline command of philosopher on multiple folders containing the annotation.txt and mzML files, it quits with the error "RemoveAll .: invalid argument.

When setting the Integrated Report step to no, it runs through without any error.

Any idea what might be the reason?

Cheers,

Frank

PS:

Here is the pipeline.yml parameter file: philosopher.yml.txt

Here is the command line output: INFO[12:22:53] Executing Pipeline v4.1.1
INFO[12:22:53] Creating workspace
WARN[12:22:53] A meta data folder was found and will not be overwritten.
INFO[12:22:53] Initiating the workspace on Bert
INFO[12:22:53] Creating workspace
WARN[12:22:53] A meta data folder was found and will not be overwritten.
INFO[12:22:53] Initiating the workspace on Ernie
INFO[12:22:53] Creating workspace
WARN[12:22:53] A meta data folder was found and will not be overwritten.
INFO[12:22:53] Annotating the database
INFO[12:22:54] Running the Database Search
MSFragger version MSFragger-3.4 Batmass-IO version 1.23.6 timsdata library version timsdata-2-8-7-1 (c) University of Michigan RawFileReader reading tool. Copyright (c) 2016 by Thermo Fisher Scientific, Inc. All rights reserved. System OS: Windows 10, Architecture: AMD64 Java Info: 1.8.0_301, Java HotSpot(TM) 64-Bit Server VM, Oracle Corporation JVM started with 35 GB memory Checking database... Checking spectral files... C:\MS_Test_Folder\Ernie\Ernie_200910_P0000_Ernie_TMTyeast_25ng_60min_R1.mzML: Scans = 15246 C:\MS_Test_Folder\Ernie\Ernie_170620_P0000_Ernie_TMTyeast_25ng_60min_R2.mzML: Scans = 15642 C:\MS_Test_Folder\Bert\Bert_200617_P0000_Bert_TMTyeast_25ng_60min_R1_20200617172353.mzML: Scans = 15498 ***FIRST SEARCH**** Parameters: num_threads = 6 database_name = C:\MS_Test_Folder\2021-11-25-decoys-contam-Saccharomyces_cerrevisiae_UP000002311_05202016_6749entries.fasta.fas decoyprefix = rev precursor_mass_lower = -20.0 precursor_mass_upper = 20.0 precursor_mass_units = 1 data_type = 0 precursor_true_tolerance = 20.0 precursor_true_units = 1 fragment_mass_tolerance = 20.0 fragment_mass_units = 1 calibrate_mass = 2 use_all_mods_in_first_search = false write_calibrated_mgf = 0 isotope_error = 0/1 mass_offsets = 0 labile_search_mode = OFF restrict_deltamass_to = all precursor_mass_mode = SELECTED localize_delta_mass = false delta_mass_exclude_ranges = (-1.5,3.5) fragment_ion_series = b,y ion_series_definitions = search_enzyme_name = Trypsin search_enzyme_sense_1 = C search_enzyme_cut_1 = KR search_enzyme_nocut_1 = P allowed_missed_cleavage_1 = 2 num_enzyme_termini = 2 clip_nTerm_M = true allow_multiple_variable_mods_on_residue = false max_variable_mods_per_peptide = 3 max_variable_mods_combinations = 5000 output_format = tsv_pepxml_pin output_report_topN = 1 output_max_expect = 50.0 report_alternative_proteins = false override_charge = false precursor_charge_low = 1 precursor_charge_high = 6 digest_min_length = 7 digest_max_length = 50 digest_mass_range_low = 500.0 digest_mass_range_high = 5000.0 max_fragment_charge = 2 deisotope = 1 deneutralloss = true track_zero_topN = 0 zero_bin_accept_expect = 0.0 zero_bin_mult_expect = 1.0 add_topN_complementary = 0 minimum_peaks = 15 use_topN_peaks = 300 minIonsScoring = 2 min_matched_fragments = 4 minimum_ratio = 0.01 intensity_transform = 0 remove_precursor_peak = 0 remove_precursor_range = -1.5,1.5 clear_mz_range_low = 125.5 clear_mz_range_high = 131.5 excluded_scan_list_file = mass_diff_to_variable_mod = 0 min_sequence_matches = 2 check_spectral_files = true variable_mod_01 = 15.99490 M 3 variable_mod_02 = 42.01060 [^ 1 variable_mod_06 = 229.162932 n^ 1 variable_mod_07 = 229.162932 S 1 add_A_alanine = 0.000000 add_C_cysteine = 57.021464 add_Cterm_peptide = 0.0 add_Cterm_protein = 0.0 add_D_aspartic_acid = 0.000000 add_E_glutamic_acid = 0.000000 add_F_phenylalanine = 0.000000 add_G_glycine = 0.000000 add_H_histidine = 0.000000 add_I_isoleucine = 0.000000 add_K_lysine = 229.162932 add_L_leucine = 0.000000 add_M_methionine = 0.000000 add_N_asparagine = 0.000000 add_Nterm_peptide = 0.0 add_Nterm_protein = 0.0 add_P_proline = 0.000000 add_Q_glutamine = 0.000000 add_R_arginine = 0.000000 add_S_serine = 0.000000 add_T_threonine = 0.000000 add_V_valine = 0.000000 add_W_tryptophan = 0.000000 add_Y_tyrosine = 0.000000 Number of unique peptides of length 7: 78837 of length 8: 75741 of length 9: 72136 of length 10: 68146 of length 11: 67093 of length 12: 62465 of length 13: 59846 of length 14: 57010 of length 15: 53296 of length 16: 51570 of length 17: 48840 of length 18: 45946 of length 19: 42987 of length 20: 41461 of length 21: 39667 of length 22: 36662 of length 23: 34966 of length 24: 33155 of length 25: 30627 of length 26: 28936 of length 27: 27448 of length 28: 26064 of length 29: 24621 of length 30: 22713 of length 31: 21183 of length 32: 19853 of length 33: 18535 of length 34: 17441 of length 35: 16122 of length 36: 14832 of length 37: 14047 of length 38: 13200 of length 39: 12119 of length 40: 11170 of length 41: 10398 of length 42: 9194 of length 43: 7752 of length 44: 5719 of length 45: 3807 of length 46: 2181 of length 47: 1137 of length 48: 519 of length 49: 253 of length 50: 114 In total 1329809 peptides. Generated 10607660 modified peptides. Number of peptides with more than 5000 modification patterns: 0 Selected fragment index width 0.10 Da. 500281306 fragments to be searched in 1 slices (7.45 GB total) Operating on slice 1 of 1: Fragment index slice generated in 14.03 s

  1. Bert_200617_P0000_Bert_TMTyeast_25ng_60min_R1_20200617172353.mzML 1.6 s | deisotoping 0.7 s [progress: 15473/15473 (100%) - 4685 spectra/s] 3.3s | postprocessing 0.2 s
  2. Ernie_170620_P0000_Ernie_TMTyeast_25ng_60min_R2.mzML 0.8 s | deisotoping 0.1 s [progress: 15584/15584 (100%) - 24426 spectra/s] 0.6s | postprocessing 0.1 s
  3. Ernie_200910_P0000_Ernie_TMTyeast_25ng_60min_R1.mzML 1.0 s | deisotoping 0.1 s [progress: 15228/15228 (100%) - 23683 spectra/s] 0.6s | postprocessing 0.1 s ***FIRST SEARCH DONE IN 0.525 MIN**
**MASS CALIBRATION AND PARAMETER OPTIMIZATION ----- --------------- --------------- --------------- --------------- MS1 (Old) MS1 (New) MS2 (Old) MS2 (New)
Run Median MAD Median MAD Median MAD Median MAD
001 0.49 0.91 -0.06 0.82 -0.66 1.26 0.02 1.18
002 2.55 1.46 -0.00 0.89 2.30 1.16 0.09 1.10
003 3.44 1.42 -0.06 0.90 0.28 1.35 0.09 1.28
----- --------------- --------------- --------------- ---------------
Finding the optimal parameters: ------- ------- ------- ------- ------- ------- ------- ------- ------- MS2 5 7 10 15 20 25 30 50
Count 6680 6703 6658 skip rest
------- ------- ------- ------- ------- ------- ------- ------- -------
------- ------- ------- ------- ------- ------- -------
Peaks 300_0 200_0 175_0 150_1 125_1 100_1
------- ------- ------- ------- ------- ------- -------
Count 6755 6760 6760 6716 skip rest
------- ------- ------- ------- ------- ------- -------
------- -------
Int. 1
------- -------
Count 6770
------- -------
------- -------
Rm P. 1
------- -------
Count 6717
------- -------

New fragment_mass_tolerance = 7 PPM New use_topN_peaks = 175 New minimum_ratio = 0.000000 New intensity_transform = 1 New remove_precursor_peak = 0 ****MASS CALIBRATION AND PARAMETER OPTIMIZATION DONE IN 2.510 MIN*****

****MAIN SEARCH**** output_format = tsv_pepXML_pin but report_alternative_proteins = 0. Change report_alternative_proteins to 1. Checking database... variable_mod_03 has an empty value. variable_mod_04 has an empty value. variable_mod_05 has an empty value. Parameters: num_threads = 6 database_name = C:\MS_Test_Folder\2021-11-25-decoys-contam-Saccharomyces_cerrevisiae_UP000002311_05202016_6749entries.fasta.fas decoyprefix = rev precursor_mass_lower = -20.0 precursor_mass_upper = 20.0 precursor_mass_units = 1 data_type = 0 precursor_true_tolerance = 20.0 precursor_true_units = 1 fragment_mass_tolerance = 7.0 fragment_mass_units = 1 calibrate_mass = 2 use_all_mods_in_first_search = false write_calibrated_mgf = 0 isotope_error = -1/0/1/2/3 mass_offsets = 0 labile_search_mode = OFF restrict_deltamass_to = all precursor_mass_mode = SELECTED localize_delta_mass = false delta_mass_exclude_ranges = (-1.5,3.5) fragment_ion_series = b,y ion_series_definitions = search_enzyme_name = Trypsin search_enzyme_sense_1 = C search_enzyme_cut_1 = KR search_enzyme_nocut_1 = P allowed_missed_cleavage_1 = 2 num_enzyme_termini = 2 clip_nTerm_M = true allow_multiple_variable_mods_on_residue = false max_variable_mods_per_peptide = 3 max_variable_mods_combinations = 5000 output_format = tsv_pepxml_pin output_report_topN = 1 output_max_expect = 50.0 report_alternative_proteins = true override_charge = false precursor_charge_low = 1 precursor_charge_high = 6 digest_min_length = 7 digest_max_length = 50 digest_mass_range_low = 500.0 digest_mass_range_high = 5000.0 max_fragment_charge = 2 deisotope = 1 deneutralloss = true track_zero_topN = 0 zero_bin_accept_expect = 0.0 zero_bin_mult_expect = 1.0 add_topN_complementary = 0 minimum_peaks = 15 use_topN_peaks = 175 minIonsScoring = 2 min_matched_fragments = 4 minimum_ratio = 0.0 intensity_transform = 1 remove_precursor_peak = 0 remove_precursor_range = -1.5,1.5 clear_mz_range_low = 125.5 clear_mz_range_high = 131.5 excluded_scan_list_file = mass_diff_to_variable_mod = 0 min_sequence_matches = 2 check_spectral_files = true variable_mod_01 = 15.99490 M 3 variable_mod_02 = 42.01060 [^ 1 variable_mod_06 = 229.162932 n^ 1 variable_mod_07 = 229.162932 S 1 add_A_alanine = 0.000000 add_C_cysteine = 57.021464 add_Cterm_peptide = 0.0 add_Cterm_protein = 0.0 add_D_aspartic_acid = 0.000000 add_E_glutamic_acid = 0.000000 add_F_phenylalanine = 0.000000 add_G_glycine = 0.000000 add_H_histidine = 0.000000 add_I_isoleucine = 0.000000 add_K_lysine = 229.162932 add_L_leucine = 0.000000 add_M_methionine = 0.000000 add_N_asparagine = 0.000000 add_Nterm_peptide = 0.0 add_Nterm_protein = 0.0 add_P_proline = 0.000000 add_Q_glutamine = 0.000000 add_R_arginine = 0.000000 add_S_serine = 0.000000 add_T_threonine = 0.000000 add_V_valine = 0.000000 add_W_tryptophan = 0.000000 add_Y_tyrosine = 0.000000 Number of unique peptides of length 7: 78837 of length 8: 75741 of length 9: 72136 of length 10: 68146 of length 11: 67093 of length 12: 62465 of length 13: 59846 of length 14: 57010 of length 15: 53296 of length 16: 51570 of length 17: 48840 of length 18: 45946 of length 19: 42987 of length 20: 41461 of length 21: 39667 of length 22: 36662 of length 23: 34966 of length 24: 33155 of length 25: 30627 of length 26: 28936 of length 27: 27448 of length 28: 26064 of length 29: 24621 of length 30: 22713 of length 31: 21183 of length 32: 19853 of length 33: 18535 of length 34: 17441 of length 35: 16122 of length 36: 14832 of length 37: 14047 of length 38: 13200 of length 39: 12119 of length 40: 11170 of length 41: 10398 of length 42: 9194 of length 43: 7752 of length 44: 5719 of length 45: 3807 of length 46: 2181 of length 47: 1137 of length 48: 519 of length 49: 253 of length 50: 114 In total 1329809 peptides. Generated 10607660 modified peptides. Number of peptides with more than 5000 modification patterns: 0 Selected fragment index width 0.03 Da. 500281306 fragments to be searched in 1 slices (7.45 GB total) Operating on slice 1 of 1: Fragment index slice generated in 12.50 s

  1. Bert_200617_P0000_Bert_TMTyeast_25ng_60min_R1_20200617172353.mzBIN_calibrated 0.2 s [progress: 15427/15427 (100%) - 11171 spectra/s] 1.4s | postprocessing 2.6 s
  2. Ernie_170620_P0000_Ernie_TMTyeast_25ng_60min_R2.mzBIN_calibrated 0.2 s [progress: 15535/15535 (100%) - 13675 spectra/s] 1.1s | postprocessing 1.9 s
  3. Ernie_200910_P0000_Ernie_TMTyeast_25ng_60min_R1.mzBIN_calibrated 0.2 s [progress: 15182/15182 (100%) - 15668 spectra/s] 1.0s | postprocessing 1.5 s MAIN SEARCH DONE IN 0.445 MIN

***TOTAL TIME 3.481 MIN**** INFO[12:26:26] Running the validation and inference on Bert INFO[12:26:26] Executing PeptideProphet on Bert
file 1: C:\MS_Test_Folder\Bert\Bert_200617_P0000_Bert_TMTyeast_25ng_60min_R1_20200617172353.pepXML processed altogether 11873 results INFO: Results written to file: C:\MS_Test_Folder\Bert\interact.pep.xml

INFO: Processing standard MixtureModel ... PeptideProphet (TPP v5.2.1-dev Flammagenitus, Build 201906281613-exported (Windows_NT-x86_64)) AKeller@ISB read in 0 1+, 8673 2+, 3079 3+, 121 4+, 0 5+, 0 6+, and 0 7+ spectra. Initialising statistical models ... Found 2685 Decoys, and 9188 Non-Decoys Iterations: .........10.........20...... WARNING: Mixture model quality test failed for charge (1+). WARNING: Mixture model quality test failed for charge (5+). WARNING: Mixture model quality test failed for charge (6+). WARNING: Mixture model quality test failed for charge (7+). model complete after 27 iterations INFO[12:26:57] Running the validation and inference on Ernie INFO[12:26:57] Executing PeptideProphet on Ernie
file 1: C:\MS_Test_Folder\Ernie\Ernie_170620_P0000_Ernie_TMTyeast_25ng_60min_R2.pepXML file 2: C:\MS_Test_Folder\Ernie\Ernie_200910_P0000_Ernie_TMTyeast_25ng_60min_R1.pepXML processed altogether 22130 results INFO: Results written to file: C:\MS_Test_Folder\Ernie\interact.pep.xml

INFO: Processing standard MixtureModel ... PeptideProphet (TPP v5.2.1-dev Flammagenitus, Build 201906281613-exported (Windows_NT-x86_64)) AKeller@ISB read in 0 1+, 18460 2+, 2865 3+, 805 4+, 0 5+, 0 6+, and 0 7+ spectra. Initialising statistical models ... Found 4366 Decoys, and 17764 Non-Decoys Iterations: .........10.........20...... WARNING: Mixture model quality test failed for charge (1+). WARNING: Mixture model quality test failed for charge (5+). WARNING: Mixture model quality test failed for charge (6+). WARNING: Mixture model quality test failed for charge (7+). model complete after 27 iterations INFO[12:27:56] Running the validation and inference on Bert INFO[12:27:56] Executing ProteinProphet on Bert
ProteinProphet (C++) by Insilicos LLC and LabKey Software, after the original Perl by A. Keller (TPP v6.0.0-rc15 Noctilucent, Build 202105101442-exported (Windows_NT-x86_64)) (no FPKM) (using degen pep info) Reading in C:/MS_Test_Folder/Bert/interact.pep.xml... ...read in 0 1+, 4419 2+, 2069 3+, 90 4+, 0 5+, 0 6+, 0 7+ spectra with min prob 0.05

Initializing 6331 peptide weights: 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100% Calculating protein lengths and molecular weights from database c:/MS_Test_Folder/2021-11-25-decoys-contam-Saccharomyces_cerrevisiae_UP000002311_05202016_6749entries.fasta.fas .........:.........:.........:.........:.........:.........:.........:.........:.........:.........1000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........2000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........3000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........4000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........5000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........6000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........7000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........8000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........9000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........10000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........11000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........12000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........13000 .........:.........:.........:.........:.........:.........:.........:.. Total: 13728 Computing degenerate peptides for 1821 proteins: 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100% Computing probabilities for 1919 proteins. Loop 1: 0%...20%...40%...60%...80%...100% Loop 2: 0%...20%...40%...60%...80%...100% Computing probabilities for 1919 proteins. Loop 1: 0%...20%...40%...60%...80%...100% Loop 2: 0%...20%...40%...60%...80%...100% Computing probabilities for 1919 proteins. Loop 1: 0%...20%...40%...60%...80%...100% Loop 2: 0%...20%...40%...60%...80%...100% Computing 1635 protein groups: 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100% Calculating sensitivity...and error tables... Computing MU for 1919 proteins: 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100% INFO: mu=2.64895e-05, db_size=12304840 INFO[12:28:00] Running the validation and inference on Ernie INFO[12:28:00] Executing ProteinProphet on Ernie
ProteinProphet (C++) by Insilicos LLC and LabKey Software, after the original Perl by A. Keller (TPP v6.0.0-rc15 Noctilucent, Build 202105101442-exported (Windows_NT-x86_64)) (no FPKM) (using degen pep info) Reading in C:/MS_Test_Folder/Ernie/interact.pep.xml... ...read in 0 1+, 12026 2+, 1832 3+, 87 4+, 0 5+, 0 6+, 0 7+ spectra with min prob 0.05

Initializing 8479 peptide weights: 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100% Calculating protein lengths and molecular weights from database c:/MS_Test_Folder/2021-11-25-decoys-contam-Saccharomyces_cerrevisiae_UP000002311_05202016_6749entries.fasta.fas .........:.........:.........:.........:.........:.........:.........:.........:.........:.........1000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........2000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........3000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........4000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........5000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........6000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........7000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........8000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........9000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........10000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........11000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........12000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........13000 .........:.........:.........:.........:.........:.........:.........:.. Total: 13728 Computing degenerate peptides for 2374 proteins: 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100% Computing probabilities for 2487 proteins. Loop 1: 0%...20%...40%...60%...80%...100% Loop 2: 0%...20%...40%...60%...80%...100% Computing probabilities for 2487 proteins. Loop 1: 0%...20%...40%...60%...80%...100% Loop 2: 0%...20%...40%...60%...80%...100% Computing probabilities for 2487 proteins. Loop 1: 0%...20%...40%...60%...80%...100% Loop 2: 0%...20%...40%...60%...80%...100% Computing probabilities for 2487 proteins. Loop 1: 0%...20%...40%...60%...80%...100% Loop 2: 0%...20%...40%...60%...80%...100% Computing probabilities for 2487 proteins. Loop 1: 0%...20%...40%...60%...80%...100% Loop 2: 0%...20%...40%...60%...80%...100% Computing probabilities for 2487 proteins. Loop 1: 0%...20%...40%...60%...80%...100% Loop 2: 0%...20%...40%...60%...80%...100% Computing 2126 protein groups: 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100% Calculating sensitivity...and error tables... Computing MU for 2487 proteins: 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100% INFO: mu=3.17999e-05, db_size=12304840 INFO[12:28:08] Integrating peptide validation
Running FPKM NSS NRS NSE NSI NSM Model EM: Computing NSS values ...

Creating 6 threads Wait for threads to finish ... ........................ done Computing NRS values ...

Creating 6 threads Wait for threads to finish ... 0--------------------------------------------------50------------------------------------------------100% ..................................................................................................... done Computing NSE values ...

Creating 6 threads Wait for threads to finish ... 0--------------------------------------------------50------------------------------------------------100% ..................................................................................................... done Computing NSI values ...

Creating 6 threads Wait for threads to finish ... 0--------------------------------------------------50------------------------------------------------100% ..................................................................................................... done Computing NSM values ...

Creating 6 threads Wait for threads to finish ... 0--------------------------------------------------50------------------------------------------------100% ..................................................................................................... done FPKM values are unavailable ... Iterations: .........done INFO[12:28:31] Creating combined protein inference
ProteinProphet (C++) by Insilicos LLC and LabKey Software, after the original Perl by A. Keller (TPP v6.0.0-rc15 Noctilucent, Build 202105101442-exported (Windows_NT-x86_64)) (no FPKM) (using degen pep info) Reading in C:/MS_Test_Folder/Bert/interact.pep.xml... ...read in 0 1+, 3886 2+, 1959 3+, 86 4+, 0 5+, 0 6+, 0 7+ spectra with min prob 0.9

Reading in C:/MS_Test_Folder/Ernie/interact.pep.xml... ...read in 0 1+, 10884 2+, 1731 3+, 73 4+, 0 5+, 0 6+, 0 7+ spectra with min prob 0.9

Initializing 9968 peptide weights: 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100% Calculating protein lengths and molecular weights from database c:/MS_Test_Folder/2021-11-25-decoys-contam-Saccharomyces_cerrevisiae_UP000002311_05202016_6749entries.fasta.fas .........:.........:.........:.........:.........:.........:.........:.........:.........:.........1000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........2000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........3000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........4000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........5000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........6000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........7000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........8000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........9000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........10000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........11000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........12000 .........:.........:.........:.........:.........:.........:.........:.........:.........:.........13000 .........:.........:.........:.........:.........:.........:.........:.. Total: 13728 Computing degenerate peptides for 1969 proteins: 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100% Computing probabilities for 2058 proteins. Loop 1: 0%...20%...40%...60%...80%...100% Loop 2: 0%...20%...40%...60%...80%...100% Computing probabilities for 2058 proteins. Loop 1: 0%...20%...40%...60%...80%...100% Loop 2: 0%...20%...40%...60%...80%...100% Computing probabilities for 2058 proteins. Loop 1: 0%...20%...40%...60%...80%...100% Loop 2: 0%...20%...40%...60%...80%...100% Computing probabilities for 2058 proteins. Loop 1: 0%...20%...40%...60%...80%...100% Loop 2: 0%...20%...40%...60%...80%...100% Computing 1797 protein groups: 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100% Calculating sensitivity...and error tables... Computing MU for 2058 proteins: 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100% INFO: mu=3.81399e-06, db_size=12304840 INFO[12:28:36] Protein inference results decoy=165 target=1632 INFO[12:28:36] Converged to 1.00 % FDR with 1296 Proteins decoy=13 threshold=0.9913 total=1309 2021/11/25 12:28:37 RemoveAll .: invalid argument

fstein commented 2 years ago

You can have a look at all files under the following link: https://oc.embl.de/index.php/s/IvjBCb9riEoZ07K

prvst commented 2 years ago

@fstein I was able to go throught the data analysis using your pipeline script, and without any issues:

INFO[13:28:20] Processing combined file                     
INFO[13:28:23] Converged to 0.99 % FDR with 7907 Peptides    decoy=79 threshold=0.907935 total=7986
INFO[13:28:23] Restoring peptide results                    
INFO[13:28:24] collecting data from individual experiments  
INFO[13:28:24] summarizing the quantification               
INFO[13:28:26] Processing combined file                     
INFO[13:28:27] Converged to 1.00 % FDR with 1296 Proteins    decoy=13 threshold=0.9913 total=1309
INFO[13:28:28] Restoring protein results                    
INFO[13:28:30] Processing spectral counts                   
INFO[13:28:31] Processing peptide counts                    
INFO[13:28:32] Processing intensities                       
INFO[13:28:32] Done  

Here's what I suggest you can try. Remove all hidden files from each data data folder, and remove the current workspace. Remove all pin files, pepindexes, and any temporary file you might have. Probably unrelated, but also change the Protein Inference option to no. This should be only used for individual analyzes, and not for combined ones.

Please, keep me posted.

fstein commented 2 years ago

Dear Felipe,

thanks for looking into this. Unfortunately, none of your suggestions worked for me (I tried removing files and starting all over again in quite some attempts) and I keep getting the same error. I even downloaded a fresh philosopher and msfragger release. Again, same error. Just to be sure, that I did not mess anything up, here is a link to the fragpipe folder that I use to start philosopher: https://oc.embl.de/index.php/s/8TaDyuU5k2d8gmH philosopher is under fragpipe\tools\philosopher

Any other idea what might be the reasons?

Cheers,

Frank

fstein commented 2 years ago

PS: Did you also try running it from scratch using my philosopher.yml file? When I did it, I had only the annotation.txt, the raw and the mzML files in the data data folders. In the data folder, I only had the fasta db (*.fas) and the philosopher.yml. All hidden files were removed.

prvst commented 2 years ago

I'm getting the same as before. I'm asking @sarah-haynes to help me pinpoint the problem, she'll try to reproduce the error on her side.

fstein commented 2 years ago

Thanks a lot. Let me know if I can help or try anything else.

prvst commented 2 years ago

@fstein can you upload your yml file?

@sarah-haynes This might be the same from #295

fstein commented 2 years ago

I did this already in my first comment. I renamed it to philosopher.yml.txt. Do you need it again?

fstein commented 2 years ago

It's also available under the following link: https://oc.embl.de/index.php/s/IvjBCb9riEoZ07K

prvst commented 2 years ago

@fstein We implemented a fix for your issue, a new version will be out soon. Thanks for reporting,

fstein commented 2 years ago

This is great news that you found a bug. Do you already have a rough idea when the updated version of philopher might be available for download?

Thanks again.

prvst commented 2 years ago

we don't have a hard date yet, but it will be soon, we just need to finish working on some other changes.

fstein commented 2 years ago

Dear Filipe,

I also tried the version that you shared with me on this particular data set. Unfortunately, I still get the same error. Or was this not bug that you mentioned not solved in the version you shared with me?

Also, in order to trigger the integrated report with Abacus, do you need to create any additional files? For example, do you need a manifest file (like with FragPipe).

prvst commented 2 years ago

You still get the RemoveALL error ?

fstein commented 2 years ago

Yes