Question: CSI:FingerID scores interpretation

[titodamiani]

Hey guys! I have a couple of questions about CSI:FingerID structure prediction scores:

1. Is there a way to interpret the CSI:FingerIDScore? I know it is hard, but I was wondering whether there are some guidelines you can recommend (e.g., the closer to 0, the better?).
2. What is the ConfidenceScore in the compound_identification.tsv summary file? Is it the COSMIC score?

I am asking because I would like to (roughly) filter out the very unreliable structure predictions from CSI-Finger:ID and keep only the “reliable” ones. Is there a way I can do it using, for example, the CSI:FingerIDScore and/or the COSMIC score? Can you guys recommend something else?

For example, in these videos (link 1, link 2) Kay says that a reliable prediction should have a tanimoto similarity between the spectrum and structure fingerprint > 50%. Could this be a way to filter out unreliable predictions? Is this tanimoto score available somewhere in the SIRIUS summary files?

Thanks for the help! Tito

[mfleisch]

Hey Tito, sorry for not responding. We somehow missed this.
I am pretty sure you have already gotten answer but just in case somebody else stumbles about this issue:

The confidence score has been designed to filter for reliable and unreliable annotations. It is important to know that this score is not an FDR but in the evaluation in our COSMIC paper we found that a confidence score of 0.64 corresponds to an approx. FDR of 10% on the evaluation data.

We used a confidence score threshold of 0.64, roughly corresponding to FDR 10% (Extended Data Fig. 9).

See, https://doi.org/10.1038/s41587-021-01045-9 or https://v6.docs.sirius-ms.io/methods-background/#COSMIC for details.