search

Nesvilab / FragPipe

A cross-platform proteomics data analysis suite
http://fragpipe.nesvilab.org
Other
184 stars 37 forks source link

Fragpipe crashes when modulating DIA-NN FDRs #1764

Closed apsteinberg closed 2 weeks ago

apsteinberg commented 3 weeks ago

Hello,

Thanks again for all your help with getting Fragpipe set up! We are attempting to do some parameter sweeps with different FDRs for DIA-NN, and are experiencing some crashes. I have attached workflows, manifests, and logs here: github_issue.zip

For the run with lower FDR (labeled q = 0.001), I assume this just means the FDR was too low and too few peptides were found to proceed? For the other runs with higher FDR (labeled q = 0.05), there are two different types of errors, and I am puzzled as to why Fragpipe crashed. It would great to get your help here with all three error cases.

Thanks for your time and help.

Best, Asher

fcyu commented 3 weeks ago

There are several different errors from three of your log files.

The first one is from MSBooster:

2024-08-31 00:08:10 [INFO] - Processing pin /data1/shahs3/users/preskaa/APS010.1_Archive/fragpipe/qvalue_0.05/argc_diaPASEF_groupFDR/A673_ArgC_F19_Slot1-37_1_2850_diatracer.pin
2024-08-31 00:08:10 [INFO] - /data1/shahs3/users/preskaa/APS010.1_Archive/fragpipe/qvalue_0.05/argc_diaPASEF_groupFDR/A673_ArgC_F19_Slot1-37_1_2850_diatracer.pin has 49186 PSMs
2024-08-31 00:08:10 [INFO] - Processing /data1/shahs3/users/preskaa/proteomics/A673_Proteogenomics/Multienzyme_MSK_Core_July_2024/ArgC/A673_ArgC_F19_Slot1-37_1_2850_diatracer.mzML
...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
2024-08-31 00:08:20 [INFO] - Not enough high quality PSMs for RT regression with escore cutoff of 3.1622776E-4. Relaxing escore cutoff to 0.01
org.apache.commons.math3.exception.NumberIsTooSmallException: bandwidth (0)
    at org.apache.commons.math3.analysis.interpolation.LoessInterpolator.smooth(LoessInterpolator.java:248)
    at org.apache.commons.math3.analysis.interpolation.LoessInterpolator.smooth(LoessInterpolator.java:395)
    at Features.StatMethods.fittingRound(StatMethods.java:284)
    at Features.StatMethods.LOESS(StatMethods.java:237)
    at Features.StatMethods.gridSearchCV(StatMethods.java:578)
    at Features.MzmlReader.setLOESS(MzmlReader.java:817)
    at Features.PercolatorFormatter.editPin(PercolatorFormatter.java:580)
    at Features.MainClass.main(MainClass.java:1318)
org.apache.commons.math3.exception.NumberIsTooSmallException: bandwidth (1)
    at org.apache.commons.math3.analysis.interpolation.LoessInterpolator.smooth(LoessInterpolator.java:248)
    at org.apache.commons.math3.analysis.interpolation.LoessInterpolator.smooth(LoessInterpolator.java:395)
    at Features.StatMethods.fittingRound(StatMethods.java:284)
    at Features.StatMethods.LOESS(StatMethods.java:237)
    at Features.StatMethods.gridSearchCV(StatMethods.java:578)
    at Features.MzmlReader.setLOESS(MzmlReader.java:817)
    at Features.PercolatorFormatter.editPin(PercolatorFormatter.java:580)
    at Features.MainClass.main(MainClass.java:1318)
2024-08-31 00:08:20 [INFO] - RT regression using 53 PSMs
Iteration 1...2...3...4...5...
2024-08-31 00:08:20 [INFO] - Regression failed, retrying with double the bandwidth: 0.02
2024-08-31 00:08:20 [INFO] - Regression failed, retrying with double the bandwidth: 0.04
2024-08-31 00:08:20 [INFO] - Best average bandwidth for mass  from grid search of 0.01,0.05,0.1,0.2 after 5 iterations is 0.04
2024-08-31 00:08:20 [INFO] - Calculating features
...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%

@yangkl96 , I think you should either let MSBooster exit with a non-zero code if this exception is fatal or not throw the except if it doesn't matter. Throwing an exception without doing anything looks dangerous.

The second one is also from MSBooster

2024-08-31 00:09:32 [INFO] - Processing /data1/shahs3/users/preskaa/proteomics/A673_Proteogenomics/Multienzyme_MSK_Core_July_2024/ArgC/A673_ArgC_F28_Slot1-46_1_2859_diatracer.mzML
umich.ms.fileio.exceptions.FileParsingException: java.util.concurrent.ExecutionException: java.lang.NullPointerException: Cannot invoke "umich.ms.datatypes.scan.impl.ScanDefault.setInstrument(umich.ms.datatypes.scan.props.Instrument)" because "this.vars.curScan" is null
    at umich.ms.fileio.filetypes.xmlbased.AbstractXMLBasedDataSource.parse(AbstractXMLBasedDataSource.java:198)
    at umich.ms.datatypes.scancollection.impl.ScanCollectionDefault.loadData(ScanCollectionDefault.java:807)
    at umich.ms.datatypes.scancollection.impl.ScanCollectionDefault.loadData(ScanCollectionDefault.java:791)
    at Features.MzmlReader.createScanNumObjects(MzmlReader.java:234)
    at Features.MzmlReader.setPinEntries(MzmlReader.java:356)
    at Features.PercolatorFormatter.editPin(PercolatorFormatter.java:335)
    at Features.MainClass.main(MainClass.java:1318)
Caused by: java.util.concurrent.ExecutionException: java.lang.NullPointerException: Cannot invoke "umich.ms.datatypes.scan.impl.ScanDefault.setInstrument(umich.ms.datatypes.scan.props.Instrument)" because "this.vars.curScan" is null
    at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
    at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:205)
    at umich.ms.fileio.filetypes.xmlbased.AbstractXMLBasedDataSource.parse(AbstractXMLBasedDataSource.java:191)
    ... 6 more
Caused by: java.lang.NullPointerException: Cannot invoke "umich.ms.datatypes.scan.impl.ScanDefault.setInstrument(umich.ms.datatypes.scan.props.Instrument)" because "this.vars.curScan" is null
    at umich.ms.fileio.filetypes.mzml.MZMLMultiSpectraParser.tagSpectrumInstarumentStart(MZMLMultiSpectraParser.java:578)
    at umich.ms.fileio.filetypes.mzml.MZMLMultiSpectraParser.call(MZMLMultiSpectraParser.java:154)
    at umich.ms.fileio.filetypes.mzml.MZMLMultiSpectraParser.call(MZMLMultiSpectraParser.java:56)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
    at java.base/java.lang.Thread.run(Thread.java:1623)
Process 'MSBooster' finished, exit code: 1
Process returned non-zero exit code, stopping

@yangkl96 could you take a look at this one too?

The third one is from EasyPQP

FileNotFoundError: [Errno 2] No such file or directory: '/data1/shahs3/users/preskaa/APS010.1_Archive/fragpipe/qvalue_0.001/aspn_diaPASEF_groupFDR/A673_AspN_F26_Slot2-31_1_2705_diatracer.psmpkl'
...
FileNotFoundError: [Errno 2] No such file or directory: '/data1/shahs3/users/preskaa/APS010.1_Archive/fragpipe/qvalue_0.001/aspn_diaPASEF_groupFDR/A673_AspN_F26_Slot2-31_1_2705_diatracer.psmpkl'

@apsteinberg It seems that something was wrong with the file system.

The last one is from MSFragger

 028 |   1.75   2.88 |   0.17   2.66 |  -0.90   4.64 |   0.02   4.55  
java.util.concurrent.ExecutionException: java.lang.NullPointerException: Cannot read field "h" because "<local30>" is null
    at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
    at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
    at e.<init>(Unknown Source)
    at edu.umich.andykong.msfragger.MSFragger.c(Unknown Source)
    at edu.umich.andykong.msfragger.MSFragger.a(Unknown Source)
    at edu.umich.andykong.msfragger.MSFragger.main(Unknown Source)
Caused by: java.lang.NullPointerException: Cannot read field "h" because "<local30>" is null
    at e.a(Unknown Source)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
    at java.base/java.lang.Thread.run(Thread.java:1623)
Process 'MSFragger' finished, exit code: 1
Process returned non-zero exit code, stopping

It seems to be a random error due to your OS because the other log shows that the files could be processed by MSFragger without any error:

*********************MASS CALIBRATION AND PARAMETER OPTIMIZATION*******************
-----|---------------|---------------|---------------|---------------
     |  MS1   (Old)  |  MS1   (New)  |  MS2   (Old)  |  MS2   (New)  
-----|---------------|---------------|---------------|---------------
 Run |  Median  MAD  |  Median  MAD  |  Median  MAD  |  Median  MAD  
 001 |   1.56   3.09 |   0.15   2.74 |  -1.01   4.66 |  -0.05   4.60  
 002 |   1.91   2.84 |   0.04   2.66 |  -0.83   4.65 |   0.00   4.57  
 003 |   2.33   2.94 |   0.04   2.74 |  -0.67   4.62 |  -0.04   4.55  
 004 |   2.33   3.14 |   0.15   2.82 |  -0.84   4.55 |  -0.04   4.44  
 005 |   2.18   3.23 |  -0.06   2.81 |  -0.74   4.67 |  -0.03   4.50  
 006 |   1.40   3.57 |   0.03   2.96 |  -1.10   4.54 |  -0.08   4.52  
 007 |  -0.17   2.74 |  -0.08   2.63 |  -1.46   5.26 |  -0.29   5.14  
 008 |   1.71   3.20 |   0.19   2.74 |  -0.90   4.76 |  -0.08   4.70  
 009 |   2.23   3.12 |   0.12   2.81 |  -0.65   4.77 |  -0.09   4.74  
 010 |   2.06   2.84 |  -0.13   2.55 |  -0.63   4.79 |  -0.08   4.65  
 011 |   1.34   2.92 |   0.16   2.70 |  -1.04   4.66 |  -0.01   4.51  
 012 |   2.04   2.89 |  -0.00   2.60 |  -0.64   4.73 |  -0.06   4.62  
 013 |   0.05   2.34 |   0.54   2.34 |  -1.65   6.45 |  -0.43   6.63  
 014 |  -0.09   3.05 |   0.09   3.14 |  -1.47   6.21 |  -0.32   6.02  
 015 |  -0.09   2.80 |  -0.13   2.84 |  -1.52   5.83 |  -0.31   5.69  
 016 |   0.37   2.77 |  -0.06   2.77 |  -1.18   5.74 |  -0.09   5.50  
 017 |   0.09   3.19 |  -0.01   3.13 |  -1.56   5.99 |  -0.36   5.85  
 018 |   0.10   2.73 |   0.14   2.69 |  -1.58   6.02 |  -0.38   5.91  
 019 |   0.10   2.35 |   0.12   2.37 |  -1.50   5.62 |  -0.53   5.46  
 020 |  -0.59   2.52 |  -0.26   2.64 |  -1.62   5.73 |  -0.29   5.64  
 021 |  -0.18   2.57 |   0.38   2.31 |  -1.52   5.41 |  -0.30   5.37  
 022 |   1.00   2.85 |  -0.06   2.65 |  -1.34   4.56 |  -0.07   4.47  
 023 |   0.00   2.66 |   0.06   2.56 |  -1.61   5.48 |  -0.20   5.37  
 024 |  -1.00   2.64 |  -0.28   2.58 |  -2.48   5.68 |  -0.45   5.60  
 025 |  -1.10   2.56 |  -0.45   2.53 |  -1.90   5.81 |  -0.28   5.92  
 026 |   1.15   2.85 |   0.03   2.63 |  -1.34   4.54 |  -0.10   4.45  
 027 |   1.94   2.88 |   0.09   2.64 |  -0.77   4.58 |  -0.03   4.58  
 028 |   1.75   2.88 |   0.17   2.66 |  -0.90   4.64 |   0.02   4.55  
 029 |   1.74   2.90 |   0.05   2.65 |  -0.96   4.53 |  -0.08   4.49  
 030 |   1.24   2.99 |  -0.09   2.79 |  -1.16   4.68 |  -0.01   4.67  
 031 |  -0.29   2.46 |   0.04   2.45 |  -1.53   5.76 |  -0.36   5.57  
 032 |   1.87   3.17 |   0.16   2.86 |  -0.73   4.66 |  -0.04   4.59  
 033 |   0.00   2.89 |   0.13   3.02 |  -0.99   5.92 |  -0.21   5.71  
 034 |  -2.39   3.42 |   0.32   3.29 |  -3.44   6.33 |  -0.99   5.94  
 035 |  -2.39   2.80 |  -0.22   2.87 |  -3.51   5.81 |  -0.51   5.36  
 036 |  -0.62   2.73 |  -0.02   2.45 |  -1.84   6.27 |  -0.17   5.92  
-----|---------------|---------------|---------------|---------------

Best,

Fengchao

apsteinberg commented 2 weeks ago

Hi Fengchao,

Thanks for the quick reply! A couple quick follow-up questions:

1) For the run with low FDR (q = 0.001), so this error does not mean that too few peptides were identified to continue?

2) For the file system errors, would you just suggest re-running? I am trying now, maybe there was some problem on our HPC at that time.

Best, Asher

fcyu commented 2 weeks ago

It is not, because it has not reached that point: DIA-NN has not started yet.

I also noticed that there is an issue when you set 0.001 to the "FDR" spin box, FragPipe somehow recorded 0: diann.q-value=0

I will fix this bug soon.

Best,

Fengchao

apsteinberg commented 2 weeks ago

Got it, thanks for explaining.

Actually, thank you so much for catching this -- it's not a problem with the spin box, I think I made a mistake in the script I wrote to edit the workflows to modulate the FDRs and it was converting 0.001 to 0. Let me try to fix this on my end, I don't think this will resolve the errors for the q = 0.05 runs, but maybe it will resolve the errors for the q = 0.001 run. I will keep you posted.

Thanks again for your help.

Best, Asher

apsteinberg commented 2 weeks ago

Actually, my apologies the script I wrote is fine and I think that you are correct that Fragpipe is writing 0 to the workflow when FDR < 0.01 but what it changing is the workflow that is found in the output directory. The initial workflow that I used as input did have diann.qvalue=0.001. This is what the beginning looks like:

# Workflow: aspn_diaPASEF_groupFDR

# Please edit the following path to point to the correct location.
# In Windows, please replace single '\' with '\\'
database.db-path=/data1/shahs3/users/preskaa/APS010.1_Archive/proteome/NDR_0.1/2024-08-05-decoys-contam-proteins.fa.f$

crystalc.run-crystalc=false
database.decoy-tag=rev_
diann.fragpipe.cmd-opts=
diann.generate-msstats=true
diann.heavy=
diann.library=
diann.light=
diann.medium=
diann.q-value=0.001
diann.quantification-strategy=3
diann.quantification-strategy-2=QuantUMS (high accuracy)
diann.run-dia-nn=true
diann.run-dia-plex=false
....

What I sent you was the workflow that is found in the fragpipe output directory. I think maybe the bug has something to do with how the workflows in the output directory are formatted, but doesn't actually change the FDR for DIA-NN. When I look at one of the successful runs I had with q-value = 0.001, the beginning of the DIA-NN log reads as follows:

DIA-NN 1.8.2 beta 8 (Data-Independent Acquisition by Neural Networks)
Compiled on Dec  1 2022 14:47:06
Current date and time: Sat Aug 31 12:20:54 2024
Logical CPU cores: 56
/admin/software/fragpipe/fragpipe-22.0/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8 --lib library.tsv --threads 20 --verbose 1 --out diann-output/report.tsv --qvalue 0.001 --matrices --no-prot-inf --smart-profiling --no-quant-files --peak-center --no-ifs-removal --report-lib-info --cfg /data1/shahs3/users/preskaa/APS010.1_Archive/fragpipe/qvalue_0.001/chymotrypsin_diaPASEF_groupFDR/filelist_diann.txt-- 

Thread number set to 20
Output will be filtered at 0.001 FDR
Precursor/protein x samples expression level matrices will be saved along with the main report
Protein inference will not be performed
When generating a spectral library, in silico predicted spectra will be retained if deemed more reliable than experimental ones
.quant files will not be saved to the disk
Fixed-width center of each elution peak will be used for quantification
Interference removal from fragment elution curves disabled
WARNING: unrecognised option [--]
DIA-NN will optimise the mass accuracy automatically using the first run in the experiment. This is useful primarily for quick initial analyses, when it is not yet known which mass accuracy setting works best for a particular acquisition scheme.

Let me know if this makes sense. I'll wait to hear back from you on the other issues, and I'm also trying to re-run now to see if some of the issues were related to file system problems as you suggested.

Thanks, Asher

apsteinberg commented 2 weeks ago

To follow-up on this, I've investigated further and the easyPQP error here:

FileNotFoundError: [Errno 2] No such file or directory: '/data1/shahs3/users/preskaa/APS010.1_Archive/fragpipe/qvalue_0.001/aspn_diaPASEF_groupFDR/A673_AspN_F26_Slot2-31_1_2705_diatracer.psmpkl'
...
FileNotFoundError: [Errno 2] No such file or directory: '/data1/shahs3/users/preskaa/APS010.1_Archive/fragpipe/qvalue_0.001/aspn_diaPASEF_groupFDR/A673_AspN_F26_Slot2-31_1_2705_diatracer.psmpkl'

... is actually not related to a file system error. It looks like this file was never created, when you go further up in the log I see this error:

Executing ['/home/preskaa/miniconda3/envs/fragpipe/bin/easypqp', 'convert', '--unimod', '/admin/software/fragpipe/fragpipe-22.0/tools/unimod_old.xml', '--max_delta_unimod', '0.02', '--max_delta_ppm', '15.0', '--fragment_types', "['b','y',]", '--enable_unannotated', '--pepxml', "['/data1/shahs3/users/preskaa/APS010.1_Archive/fragpipe/qvalue_0.001/aspn_diaPASEF_groupFDR/interact-A673_AspN_F26_Slot2-31_1_2705_diatracer.pep.xml']", '--spectra', '/data1/shahs3/users/preskaa/proteomics/A673_Proteogenomics/Multienzyme_MSK_Core_July_2024/AspN/A673_AspN_F26_Slot2-31_1_2705_diatracer.mzML', '--exclude-range', '-1.5,3.5', '--psms', 'A673_AspN_F26_Slot2-31_1_2705_diatracer.psmpkl', '--peaks', 'A673_AspN_F26_Slot2-31_1_2705_diatracer.peakpkl']
Warning: OPENMS_DATA_PATH environment variable not found and no share directory was installed. Some functionality might not work as expected.
2024-08-30 23:21:46 - Info: Converting ['/data1/shahs3/users/preskaa/APS010.1_Archive/fragpipe/qvalue_0.001/aspn_diaPASEF_groupFDR/interact-A673_AspN_F26_Slot2-31_1_2705_diatracer.pep.xml'].
2024-08-30 23:21:46 - Info: Parsing run A673_AspN_F26_Slot2-31_1_2705_diatracer.
2024-08-30 23:21:46 - Info: Parsing pepXML: /data1/shahs3/users/preskaa/APS010.1_Archive/fragpipe/qvalue_0.001/aspn_diaPASEF_groupFDR/interact-A673_AspN_F26_Slot2-31_1_2705_diatracer.pep.xml
2024-08-30 23:21:46 - Info: 3566 PSMs parsed.
2024-08-30 23:21:46 - Info: Generate theoretical spectra.
2024-08-30 23:21:46 - Info: Processing spectra from file /data1/shahs3/users/preskaa/proteomics/A673_Proteogenomics/Multienzyme_MSK_Core_July_2024/AspN/A673_AspN_F26_Slot2-31_1_2705_diatracer.mzML.
/opt/conda/conda-bld/openms-meta_1716538752609/work/src/openms/source/FORMAT/HANDLERS/XMLHandler.cpp(104): While loading '/data1/shahs3/users/preskaa/proteomics/A673_Proteogenomics/Multienzyme_MSK_Core_July_2024/AspN/A673_AspN_F26_Slot2-31_1_2705_diatracer.mzML': comment or processing instruction expected( in line 1382945 column 15)
Traceback (most recent call last):
  File "/home/preskaa/miniconda3/envs/fragpipe/bin/easypqp", line 8, in <module>
    sys.exit(cli())
  File "/home/preskaa/miniconda3/envs/fragpipe/lib/python3.9/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/preskaa/miniconda3/envs/fragpipe/lib/python3.9/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/home/preskaa/miniconda3/envs/fragpipe/lib/python3.9/site-packages/click/core.py", line 1719, in invoke
    rv.append(sub_ctx.command.invoke(sub_ctx))
  File "/home/preskaa/miniconda3/envs/fragpipe/lib/python3.9/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/preskaa/miniconda3/envs/fragpipe/lib/python3.9/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/preskaa/miniconda3/envs/fragpipe/lib/python3.9/site-packages/easypqp/main.py", line 90, in convert
    psms, peaks = conversion(pepxmlfile_list, spectralfile, unimodfile, exclude_range, max_delta_unimod, max_delta_ppm, enable_unannotated, enable_massdiff, fragment_types, fragment_charges, enable_specific_losses, enable_unspecific_losses, max_psm_pep, precision_digits)
  File "/home/preskaa/miniconda3/envs/fragpipe/lib/python3.9/site-packages/easypqp/convert.py", line 1051, in conversion
    input_map = get_map_mzml_or_mzxml(spectralfile, 'mzml')
  File "/home/preskaa/miniconda3/envs/fragpipe/lib/python3.9/site-packages/easypqp/convert.py", line 1033, in get_map_mzml_or_mzxml
    fh.transform(path, consumer)
  File "pyopenms/_pyopenms_12.pyx", line 2200, in pyopenms._pyopenms_12.MzMLFile.transform
  File "pyopenms/_pyopenms_12.pyx", line 2286, in pyopenms._pyopenms_12.MzMLFile._transform_1
RuntimeError: - due to that error of type Parse Error in: /opt/conda/conda-bld/openms-meta_1716538752609/work/src/openms/source/FORMAT/HANDLERS/XMLHandler.cpp@105-void OpenMS::Internal::XMLHandler::fatalError(ActionMode, const OpenMS::String&, OpenMS::UInt, OpenMS::UInt) const

Not sure if this is related to how the openMS installation is working as there is this warning, or something else entirely.

Thanks again for your help.

Best, Asher

fcyu commented 2 weeks ago

When I look at one of the successful runs I had with q-value = 0.001, the beginning of the DIA-NN log reads as follows: Let me know if this makes sense.

Yes, it makes sense. Glad to know that this bug just affects the GUI mode and the fragpipe.workflow, and won't affect the actual running.

... is actually not related to a file system error. It looks like this file was never created, when you go further up in the log I see this error

Since the other EasyPQP convert commands executed successfully, I don't think it was due to the OpenMS or EasyPQP installation.

/opt/conda/conda-bld/openms-meta_1716538752609/work/src/openms/source/FORMAT/HANDLERS/XMLHandler.cpp(104): While loading '/data1/shahs3/users/preskaa/proteomics/A673_Proteogenomics/Multienzyme_MSK_Core_July_2024/AspN/A673_AspN_F26_Slot2-31_1_2705_diatracer.mzML': comment or processing instruction expected( in line 1382945 column 15)

It seems that OpenMS couldn't load that mzML file.

Best,

Fengchao

apsteinberg commented 2 weeks ago

Sounds good. For the mzML file error I guess maybe there is something wrong with the the DIA input file then? I can just exclude it from the re-run.

Thanks, Asher

fcyu commented 2 weeks ago

I am not sure if it is about the input file because other modules analyzed the data without any error. Maybe some file system issues happened at random.

Best,

Fengchao

apsteinberg commented 2 weeks ago

Hmm okay, so you would suggest just re-running? Is this type of behavior common? I am a bit surprised if this is the case as I have not experienced this before on our HPC.

Thanks, Asher

fcyu commented 2 weeks ago

It is not common. This is my first time seeing this error.

Best,

Fengchao

apsteinberg commented 2 weeks ago

Got it, okay this is good to know. It does seem that simply re-running fixed the issue though with that run. The last run that I'm having trouble with is this one with the log, manifest, and workflow attached here:

240902_fragpipe_issue.zip

It seems there is an MSBooster error, but not sure what the message indicates I should do to resolve it:

2024-09-01 06:57:10 [INFO] - Processing /data1/shahs3/users/preskaa/proteomics/A673_Proteogenomics/Multienzyme_MSK_Core_July_2024/Trypsin/Trypsin_50_80_Slot2-26_1_1514_diatracer.mzML
...10%...20%...30%...40%...50%...60%...70%...80%...90%java.util.concurrent.ExecutionException: java.lang.NullPointerException: Cannot invoke "java.lang.Comparable.compareTo(Object)" because "k1" is null
    at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
    at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
    at Features.MzmlReader.setPinEntries(MzmlReader.java:390)
    at Features.PercolatorFormatter.editPin(PercolatorFormatter.java:335)
    at Features.MainClass.main(MainClass.java:1318)
Caused by: java.lang.NullPointerException: Cannot invoke "java.lang.Comparable.compareTo(Object)" because "k1" is null
    at java.base/java.util.TreeMap.compare(TreeMap.java:1569)
    at java.base/java.util.TreeMap$NavigableSubMap.<init>(TreeMap.java:1646)
    at java.base/java.util.TreeMap$AscendingSubMap.<init>(TreeMap.java:2169)
    at java.base/java.util.TreeMap.subMap(TreeMap.java:1190)
    at umich.ms.fileio.filetypes.xmlbased.AbstractXMLBasedDataSource.parse(AbstractXMLBasedDataSource.java:129)
    at umich.ms.datatypes.scancollection.impl.ScanCollectionDefault.loadData(ScanCollectionDefault.java:807)
    at umich.ms.datatypes.scancollection.impl.ScanCollectionDefault.loadData(ScanCollectionDefault.java:791)
    at Features.MzmlReader.getScanNumObject(MzmlReader.java:248)
    at Features.MzmlReader$setScanNumPepObj.run(MzmlReader.java:341)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:577)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
    at java.base/java.lang.Thread.run(Thread.java:1623)
Process 'MSBooster' finished, exit code: 1
Process returned non-zero exit code, stopping

Thanks again for your time and help.

Best, Asher

fcyu commented 2 weeks ago

Hi Asher,

Glad to hear that rerunning FragPipe resolves the issue, which indicates that it was indeed due to some random OS or file system errors.

Regarding the MSBooster error, it is similar to the previous one: https://github.com/Nesvilab/FragPipe/issues/1764#issuecomment-2323066677 . I will let Kevin @yangkl96 take a look.

Best,

Fengchao

apsteinberg commented 2 weeks ago

Great, sounds good -- thank you both for all your help!

Best, Asher

yangkl96 commented 2 weeks ago

Hi Asher,

MSBooster is having issues parsing specific mzml files. Could you please share the following files with me at this DropBox link?

Thanks, Kevin

apsteinberg commented 2 weeks ago

Hi Kevin,

Absolutely. Let me just check with my supervisor if it's okay to share the files. Also, in the meantime I think I may have realized what the issue is here, and I apologize for not spotting this sooner. I have been launching multiple Fragpipe runs on our HPC which have the same manifest yet different workflows (i.e., the different q-values). In the workflow it is running diatracer which is generating .mzML files for each of the diaPASEF files in the manifest. I have a strong suspicion this is part of the problem. Is there a way I can have Fragpipe output the .mzML files from each of the runs to different directories so these runs don't interfere with each other and I can still run in parallel?

I could alternatively just duplicate the diaPASEF files for each run, but this will triple my footprint on our HPC.

Thanks again, and I will get back to you about sharing these files ASAP.

Best, Asher

fcyu commented 2 weeks ago

Hi Asher,

Yeah, running multiple FragPipe instances for the same input files will cause problems because FragPipe generate temp files to the folder where the input files are.

Is there a way I can have Fragpipe output the .mzML files from each of the runs to different directories so these runs don't interfere with each other and I can still run in parallel?

With the current version, no. An easy workaround is creating softlinks of the mzML files to your result folder and then loading those softlinks to FragPipe. I will let FragPipe write the temp files directly to the result folder in the future.

Best,

Fengchao

apsteinberg commented 2 weeks ago

Hi Fengchao,

Okay understood about the multiple FragPipe instances. For the soft links, are you suggesting that I create a new input directory, then fill it with soft links to the diaPASEF files in the actual input directory?

For example, if I have a diaPASEF file here:

/data1/shahs3/users/preskaa/proteomics/A673_Proteogenomics/Multienzyme_MSK_Core_July_2024/Trypsin/Trypsin_100_10_Slot2-29_1_1516.d

Create a soft link named Trypsin_100_10_Slot2-29_1_1516.d in a new directory named Trypsin_qvalue_0.001

and in the manifest write:

/Trypsin_qvalue_0.001/Trypsin_100_10_Slot2-29_1_1516.d  A       DIA

Also, @yangkl96 I just emailed with a link to the mzML and pin files as well.

Thanks, Asher

fcyu commented 2 weeks ago

For the soft links, are you suggesting that I create a new input directory, then fill it with soft links to the diaPASEF files in the actual input directory?

Correct.

Best,

Fengchao

apsteinberg commented 2 weeks ago

Sounds good, I will give this a try today and get back to you asap.

Thanks, Asher

yangkl96 commented 2 weeks ago

I have fixed the issue with A673_ArgC_F19_Slot1-37_1_2850_diatracer.mzML, where the RT calibration was failing.

I forgot to ask, can I also have A673_ArgC_F28_Slot1-46_1_2859_diatracer.mzML and its associated PIN file?

Thanks, Kevin

apsteinberg commented 2 weeks ago

Great! Sure, I can share that with you. Would you prefer if I give what Fengchao suggested a try first? I'm wondering if the problems with the mzML files are due to what we discussing, as they are being generated by Fragpipe in the diatracer step. I should be able to do it today.

Thanks, Asher

yangkl96 commented 2 weeks ago

Yes, you can try that first. I actually wasn't able to reproduce one of the mzml parsing errors in your log, so it's very possible resolving the mzml generation issue may also resolve the MSBooster issues

apsteinberg commented 2 weeks ago

Sounds good, thanks @yangkl96. I was able to re-run everything by creating a new input directory filled with symlinks for each Fragpipe run that was running in parallel on the HPC, and this resolved the issue. Thank you both for all your time and help on this one. If I may suggest, it could be useful to issue a warning to users when there are multiple runs working with the same set of input files in parallel; though I will add that I think what we are doing may be a bit abnormal.

Thanks again.

Best wishes, Asher