Closed badisa closed 4 months ago
Would you mind adding these changes (+ either the plot or some other convenient export of the resulting timings) to test_benchmark_free_energy.py ? Will be good to track timings across this grid of conditions in the future.
Added in https://github.com/proteneer/timemachine/pull/1287/commits/19b5c7d655e32c6ad2b3492717f0021dc9591298
Didn't change the tests as the settings can take several hours to run and doesn't seem worth it to add to the nightly tests for just timings. This now more closely mirrors our logic around the other benchmarks: https://github.com/proteneer/timemachine/blob/9c023081c661ec320d2198d8f766257b5d13debf/tests/test_benchmark.py#L524-L533
Out of curiosity I ran vacuum and solvent over night going out to 2000 frames to see if the performance differences changed. Only noticeable difference was that with solvent HREX only ~20%~ 30% of ns/day was lost versus 65% lost in vacuum. Which makes sense as solvent is more likely to be dominated by the cost of MD rather than by the cost of whatever work these methods do on their own.
Benchmarks
The reference refers to what is currently on master, the comparison is this branch. Generated using the test here: https://github.com/proteneer/timemachine/blob/1b8f07f102cdeab76f5b7e2126dcd8ba7bfcc49e/tests/test_benchmark_free_energy.py#L76 with the additional change to run with 2, 4, 8, 16, 32 and 48 windows as well as making sure every window always ran 100 frames.
Run on an A10 GPU with Cuda Arch 8.6. Only sampled once, but think the differences are significant even without replicates.
Also ran a small graph (bisection + hrex + water sampling) that was about 5% faster with this change.
Vacuum
Doesn't make a difference on vacuum simulations, presumably the cost of constructing potentials is cheap (no neighborlist notably, tiny systems, etc)
Solvent
Seems to make the biggest difference for bisection
Complex
Without water sampling
With water sampling