samplchallenges / SAMPL5_logD_PredictionAnalysis

Analysis of all predictions submitted for the distribution coefficient part of the SAMPL5 challenge
MIT License
2 stars 1 forks source link

Attaching feedback from Diogo Santos Martins #14

Closed davidlmobley closed 8 years ago

davidlmobley commented 8 years ago

comments_overview_SAMPL5_DC_diogo_jun27.pdf

bannanc commented 8 years ago

I'm adding shorted comments for the suggestions so we don't have to open a pdf to see them.

bannanc commented 8 years ago
  1. Over estimation of the magnitude of logD.

He fits calculated values to experiment and plots slope and y-intercept (color coding for R values). No one has slopes greater than 0.5. (I think he must have done this in the reverse of our plots because most people had slopes greater than 1 in the plots I've looked at).

You did actually address this in your talk at D3R, but it was in the part about Christopher Fennell's results (he found that if he divided his results by 2 he got significantly better results).

bannanc commented 8 years ago

2) 23 and 49 use similar methods and are therefore not representative sets of predictions

I did check who submitted the example plots, I just picked two that were in the middle by all the error metrics. We probably should make sure they aren't from the same methods I guess.

bannanc commented 8 years ago

3) Minor changes:

a) TIP3P water b) effect instead of affect on figure 1 title c) use PME and reaction field for the legend in figure 1 d) figure 7 close parenthesis

davidlmobley commented 8 years ago

Yes, check point #2, though "similar" may not mean "the same". If you want to update with what they did I can comment on whether I think they're too similar or not.

In terms of overestimation of the magnitude -- maybe just quickly remind me where we address this and what we say about it (or point me to page whatever #) and I'll see if I think we need more caveats/to expand discussion.

bannanc commented 8 years ago

Changes included in pull requests #19 and #20