Reviewer 3 Comment 1

[ramess101]

@mrshirts @jpotoff @msoroush

In figure 2, the authors perform small local variations to identify a slightly shifted local minima for the different molecules. Have the authors considered performing larger variations in the scaling factor for the well-depth to check if there is an even lower minima in the scoring (error) function? It may be useful to also show the sensitivity of these plots to reasonable variations in the scoring function. (Let's say vary the distribution of weights between liquid density and vapor pressure by about 10%).

Two separate issues raised here.

First, regarding the individualized scan over a wider range to find an even lower minima. Because this analysis does not modify sigma or lambda, the scoring function just continues to increase with large deviations in the scaling factor. We can see this clearly in the heat maps for cyclohexane where the errors just continue to get worse over a wide range of epsilon for a fixed sigma/lambda.

Second, although it would be very simple to play around with the scoring function and it could be insightful from an optimization standpoint, I think it would distract from the main purpose of this analysis, namely, MBAR provides very smooth optimization valleys when varying a single scaling factor and that this scaling factor is typically close to 1. If we changed the scoring function we would no longer have any reason to hope for psi close to 1 because the transferable and individualized models would have different targets.

[jpotoff]

@ramess101 Agree with that assessment. Regarding item 1, we've already done enough brute force calculations to know there isn't a magical region hiding out there with better parameters. For example, we did try to optimize the C parameter starting with the NERD parameters, which have a very different epsilon and sigma and came back to the more TraPPE like epsilon and sigma of the Mie potential. The results were bad enough that there was no need to generate a heat map to know they weren't good.

Regarding the weighting of the scoring function, changing the weights is largely irrelevant. The weights have been set by use to give us the model we want. It's pretty obvious that if one changes the weights, one would get a different set of "optimum" parameters, with a different reproduction of the various physical properties.

[ramess101]

@jpotoff

Agreed.

[ramess101]

@mrshirts @jpotoff @msoroush