bjornwallner / DockQ

DockQ is a single continuous quality measure for protein docked models based on the CAPRI evaluation protocol
MIT License
185 stars 46 forks source link

DockQ.py incorrectly calculating iRMS #16

Closed kaymccoy closed 2 months ago

kaymccoy commented 1 year ago

I’m attempting to get the DOCKQ score of a model of CAPRI target #47, from the score_set dataset. The model is named Target47_0064.pdb and the correct crystal structure is named Target47_3u4e.pdb. However, the DOCKQ script seems to be getting the interface RMSD wrong. Both are attached (but with the extention txt added, as pdb files aren't allowed to be uploaded by github) here:

Target47_0064.pdb.txt Target47_3u4e.pdb.txt

In the DockQ folder, I’ve run

scripts/fix_numbering.pl /path/to/Target47_0064.pdb /path/to/Target47_3u4e.pdb

then

python3 DockQ.py /path/to/Target47_0064.pdb.fixed /path/to/Target47_3u4e.pdb

which produces the following output (truncated to just show Fnat through DockQ):

Fnat 0.833 45 correct of 54 native contacts
Fnonnat 0.297 19 non-native of 64 model contacts
iRMS 4.576
LRMS 1.827
DockQ 0.629 

However when I calculate the three components of DockQ myself (using protein structure tools from the C++ library Mosaist), I find:

Fnat: 0.846
iRMS: 1.019
lRMS: 1.819
DockQ: 0.829

The slight differences in Fnat and lRMS aren’t very concerning to me (I assume they come down to some slight difference in atom-matching between the structures), but the iRMS is significantly off. Examining the structure visually, it seems like the iRMS should be around 1 Å, so I think this probably comes down to a bug in the DOCKQ script where it sometimes gets iRMS wrong. This happens reproducibly with a number of other models for Target 47 as well, all having larger iRMS values than they should, lowering their DockQ scores considerably (usually a difference of ~0.2). Here's a visual of the model and crystal structure aligned in pymol, with the crystal in yellows, model in greens, and different binding partners in either lighter or darker shades. My apologies if I'm wrong, but it appears the iRMS should be much lower than 4.5 Å, and an iRMS of around 1.0 Å would be reasonable. I figured I should bring the discrepancy here, in case it represents some buggy edge case.

Screen Shot 2023-01-10 at 3 20 20 PM
clami66 commented 2 months ago

Hi, please check the latest release (v2.0) that should fix this bug. Will close this for now, let me know if you have more questions

kaymccoy commented 2 months ago

Thanks so much! I really appreciate the update.