FreshAirTonight
commented
1 year ago Hello,
iScore stands for the interface score. They are the same metric. piTMs is the predicted TM-score of protein-protein interface residues. The two metrics, iScore and piTMs, are different. In practice, iScore is better and is the main metric we have adopted.
Both iScore and piTMs have its origin in the iAlign work you mentioned, but they serve very different purposes from the previous scoring metrics appeared in the iAlign paper, in which we proposed the interface TM-score (iTM-score) and the Interface Similarity score (IS-score) for measuring the similarity between two protein-protein interfaces (from known complex structures). By contrast, in the case of AF2Complex, iScore and piTMs are used for estimating the confidence of a single protein-protein interface model (from its unknown experimental structure target). As such, the p-values derived for the IS-score in iAlign should not be applied to iScore or piTMs. Their statistical significance has to be calculated separately, as we detailed in our eLife work using statistics from ~7,000 putatively non-interacting protein pairs from E. Coli. If we use this ~7,000 set as the random background, an iScore of 0.4 has an estimated p-value of ~0.01, which one may consider significant.
With respect to your complex target, piTMs of 0.42 and IS-score of 0.1 are not considered significant. This could be interpreted as a miss by AF2Complex, if you have strong experimental evidence that they do interact. It is possible that the structural model you obtained is still a good model, but the program is uncertain about its quality. You may want to try multiple runs with different conditions, such as different MSA depth or pairing modes. It is not uncommon that a single confident model magically appears in over hundreds of models.
ZKrekhno
Hi,
I am having some confusion figuring which metric to use for assessing model confidence. In your eLife paper, you describe the iScore metric for ranking model confidence. Does it have any relationships with interface score or piTMs that get output by the standard AF2Complex pipeline, or is that a new metric? Should that metric take precedence over the IS or piTMs for assessing the results of AF2Complex?
Also, to follow up to that, is there a reason to choose piTMs or IS-score for interpretation of the model output, and is the p-value to score relationship still roughly the same to the one derived for iAlign in the Bioinformatics, 2010 paper? How often do these metrics give significantly different results per model? We have tried modelling an interaction between two trans-membrane proteins (that we are fairly certain interact directly) and got the piTMs of 0.42, and IS-score of 0.1, and are a bit confused on how to interpret that. Would this be classified as medium confidence interaction by AF2Complex metrics or not?