openvax / mhcflurry

Peptide-MHC I binding affinity prediction
http://openvax.github.io/mhcflurry/
Apache License 2.0
191 stars 57 forks source link

Issue with qualitative measure dictionary in code VS paper #167

Closed BaptisteGrs closed 4 years ago

BaptisteGrs commented 4 years ago

Dear mhcflurry team,

I'm currently working on mhc-binding predictors and your tool has been really interesting to study so far. There is something that I'm not sure to understand however and I'd like to know if you could help me. In the downloads-generation/data_curated/curate.py function, the dictionary to assign quantitative values for "qualitative" assays is as such :

QUALITATIVE_TO_AFFINITY_AND_INEQUALITY = { "Negative": (5000.0, ">"), "Positive": (50.0, "<"), # used for mass-spec hits "Positive-High": (100.0, "<"), "Positive-Intermediate": (1000.0, "<"), "Positive-Low": (5000.0, "<"), }

The value for "Positive" reference is set to 50 nM, while 500 is indicated in the paper :

"Quantitative affinity data is associated with an inequality of (=). For qualitative affinity data, we assigned the following inequalities and measurement values:positive-high, < 100 nM;positive, < 500 nM,positive-intermediate, < 1,000 nM;positive-low, < 5,000 nM;negative, > 5,000 nM. In theMHCflurry(train-MS) variant, MS-identified ligands are assigned the value ‘‘< 500 nM."

I'm not sure if I am not understanding something or if this is a simple typo. Thanks for your help! I look forward to your response,

Best, Baptiste

timodonnell commented 4 years ago

Good catch! You're right - in the 2018 cell systems paper describing mhcflurry 1.2, we used 500 nM. For the current latest version (1.6, described in the 2020 preprint) this switched to a 50 nM threshold, which let the mass spec data more strongly guide the predictions and gave better results in some of our tests. This is why you're seeing a number there that doesn't match the 2018 paper.

However, we've since found that the 1.6 version has some issues with affinity calibration, and so we'll most likely be switching back to 500 nM for version 1.7, which should be released in the coming weeks. Here's the branch where that is happening: https://github.com/openvax/mhcflurry/tree/1.7.0b

Hope that helps.

Tim

timodonnell commented 4 years ago

Noting that in version 2.0.0, this value has been switched to "< 100 nM".