Question about phosphosite localization confidence.

[andyzim111]

Hi, Vadim

I recently want to test DIA-NN function of phosphosite localization. I downloaded the raw MS data which included 166 synthetic phosphopeptides spiked in yeast lysate and the spectral library generated by spectronaut, which is published in Nature Communications entitled "A data-independent acquisition-based global phosphoproteomics system enables deep profiling".

Based on the provided spectral library that contained 166 phosphopeptides, DIA-NN identified 103 phosphopeptides with PTM location confidence >0.75 out of total 166 phosphopeptides.

Considering relatively low percentage (103/166) phosphopeptides with confident localization, I wonder what goes wrong with my analysis.

In the study, the authors used spectronaut to analyze the same data, resulting in 157 class I phosphopeptides.

Thanks. 41467_2021_22759_MOESM5_ESM.xlsx spectronaut_library_DIA-NN.txt Sythetic phosphopeptides.txt

[vdemichev]

I don't think this percentage is low... The localisation score DIA-NN reports corresponds to confidence that all phosphosites on the peptide are correct. I don't trust confidence scores reported by Spectronaut, there hasn't been any validation of these published.

Best, Vadim

[andyzim111]

Thanks for your reply.

I noticed that multiple phosphorylated peptides has only one PTM localization confidence in DIA-NN reports, and then which one phosphosite does the confidence correspond to?

[vdemichev]

Localisation score in DIA-NN indicates the probability that all phosphosites on the respective precursor are shown correctly, i.e. if shown unmodified, then are unmodified, and if shown modified, then a re modified.

[andyzim111]

I got it. The localisation score shows the likeliness that the whole phosphopeptide bearing modifications is present or not, this kind of like "peptidoform".

[oliver-bernhardt]

I don't think this percentage is low... The localisation score DIA-NN reports corresponds to confidence that all phosphosites on the peptide are correct. I don't trust confidence scores reported by Spectronaut, there hasn't been any validation of these published.

Best, Vadim

Here is the paper: https://www.nature.com/articles/s41467-020-14609-1

You know about it, you cited it in your own work. The localization algorithm is explained in detail in the supplementary and it was validated in this work alone using 3 different methods.

[vdemichev]

Hi Oliver @oliver-bernhardt, of course I've read the paper, was highly interesting, very nice contribution by you and the team! I trust some things in Spectronaut, but I've no idea how well your localisation algorithm actually works, as I don't think the validation in that paper is sufficient.

Therein it's validated on a single dataset of ~200 detectable spike-in peptides, and

• You yourself report 1.7% error rate at 0.99 reported confidence and 0.9% error rate for 0.999 confidence (should be 0.1% instead) - means calculation of FDR is erroneous (Supp. Fig. 2).
• Not clear if the results on spike-in peptides will translate to the yeast background peptides. Suppose almost all of your spike-in peptides that are detectable at a given dilution are as abundant as top 1% of background peptides. Then they would all be likely localised correctly. Does not imply at all correctness for background peptides. This is an exagerration, but I think illustrates the point why a different validation method is necessary for confidence.
• As described, Spectronaut uses a hard-coded formula for localisation confidence, not target-decoy-type method. How can we be sure it performs similarly on data from different mass specs (which have different noise levels) / different gradients / different injection amounts / different backgroud?

[oliver-bernhardt]

So instead of: "there hasn't been any validation of these published."

you should rather say that you don't agree on how it was validated.

[vdemichev]

Well, if a 'validation' experiment does not really validate the algorithm, I would not call it 'validation'. Does not mean it's not useful: it does show that filtering based on localisation score can be used to give confidence in localisation. But it does not prove that the confidence rate reported by Spectronaut is accurate.

[oliver-bernhardt]

while we are at the topic, can you point me to the publication where the PTM.Site.Confidence and the PTM.Qvalue reported in DIA-NN is described and validated?

[vdemichev]

PTM.Q.Value is described and validated in our ubiquitin paper https://www.nature.com/articles/s41467-021-25454-1, supp. fig. 10. Which also includes a benchmark for Spectronaut, btw. Although the implementation is now somewhat different. We did not publish benchmarks for site confidence.

[oliver-bernhardt]

Just to clarify but that means that the recommendation is to use PTM.Q.Value for localization filtering? Since PTM.Site.Confidence is not validated right now.

[vdemichev]

All recommendations are in DIA-NN docs. Not all validations have been published.