search

MobleyLab / alchemical-analysis

An open tool implementing some recommended practices for analyzing alchemical free energy calculations
MIT License
120 stars 60 forks source link

multicore / gpu compataibility #91

Open raziel81 opened 8 years ago

raziel81 commented 8 years ago

Hi there, I noticed that alchemical analysis only uses a single core at a time and doesn't utilize gpus at all. I'm wondering if their is any intent of adding this to the code in the near future? Generally speaking, the options are numerous - multi core/multi thread/ gpus and there are many different strategies within python that can be implemented like autojit and cython. If there aren't any such plans in the moment, can the addition of any of these enhancements damage the code or the program's accuracy ?

All the best

mrshirts commented 8 years ago

My general though is "never optimize prematurely". Are there specific applications where running the free energy analysis is a bottleneck?

raziel81 commented 8 years ago

specifically, i'm computing relatively big data sets (~40 GBs of samples) via alchemical analysis. I'm using an intel hexacore i7 with 32GB and an Nvidia 780 Ti GPU and it seems like a tremendous waste of time just to wait on a single core calculation. I won't mind making my own efforts in this matter, but I wanted to be sure that any such modifications (even just using cython to compile some of the code) will not modify potential results as far as the developers are aware (although their really is just one way to know this for sure).

On Wed, Aug 24, 2016 at 10:35 PM, Michael Shirts notifications@github.com wrote:

My general though is "never optimize prematurely". Are there specific applications where running the free energy analysis is a bottleneck?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/MobleyLab/alchemical-analysis/issues/91#issuecomment-242279005, or mute the thread https://github.com/notifications/unsubscribe-auth/AOTcZhBhGOhu470LcLeBRGVHDroibraMks5qjRumgaJpZM4Jsq4- .

davidlmobley commented 8 years ago

A couple of questions on this:

  1. Is the issue really/primarily alchemical-analysis, or is it pymbar? i.e. if you make changes to alchemical-analysis but pymbar doesn't/can't utilize GPUs/multiple cores then you'll still be out of luck.
  2. Are your datasets really large because you've run such long free energy calculations (or with so many lambda windows) because you really need that many data points, or are they large because you've saved far more data points than you need? (i.e. see discussion in #90 )

We don't have any immediate plans to deal with this. Also, the code itself will need some larger architectural changes (i.e. see the open PR from Loeffler) in the relatively near future that I hope to get to later this fall (or get help with making -- but I have some deadlines to get through first), so I'd recommend against doing anything which would require extensive code changes at this point as it would likely break when we do those architectural changes. However, if you want to get involved with the architectural changes we'd probably be supportive and it might provide you an opportunity to try out the optimizations you're discussing.

raziel81 commented 8 years ago

1) i'm just using the alchemical analysis script and as far as i can tell, when i use top it just shows "python" as the active process so I can't say for certain if its pymbar or alchemical analysis. I'll be glad to give you more detailed information if you can suggest a way to obtain it. 2) at this point its hard for me to be certain if i have enough information or not for my case of interest, as i have no experimental data to compare to in practice. The all-atom representations of the structures i'm looking into consist of over 100 atoms while in CG representation they are just 6 beads so under-sampling is a big concern which is why i'm using such big data sets in the first place.

Thanks for your input in this matter, I will look forward then for the aforementioned changes when they come out. Generally i'd be glad to test such optimizations at the time of your convenience. Regardless, i do think that just using a c++ compilation scheme within the python script would most likely have the least effect on the code (would essentially be using it "as is", only compile via c++ instead of interpret it via python) while offering a decent amount of acceleration to the process as a whole.

On Thu, Aug 25, 2016 at 9:59 AM, David L. Mobley notifications@github.com wrote:

A couple of questions on this:

  1. Is the issue really/primarily alchemical-analysis, or is it pymbar? i.e. if you make changes to alchemical-analysis but pymbar doesn't/can't utilize GPUs/multiple cores then you'll still be out of luck.
  2. Are your datasets really large because you've run such long free energy calculations (or with so many lambda windows) because you really need that many data points, or are they large because you've saved far more data points than you need? (i.e. see discussion in #90 https://github.com/MobleyLab/alchemical-analysis/issues/90 )

We don't have any immediate plans to deal with this. Also, the code itself will need some larger architectural changes (i.e. see the open PR from Loeffler) in the relatively near future that I hope to get to later this fall (or get help with making -- but I have some deadlines to get through first), so I'd recommend against doing anything which would require extensive code changes at this point as it would likely break when we do those architectural changes. However, if you want to get involved with the architectural changes we'd probably be supportive and it might provide you an opportunity to try out the optimizations you're discussing.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/MobleyLab/alchemical-analysis/issues/91#issuecomment-242441287, or mute the thread https://github.com/notifications/unsubscribe-auth/AOTcZl6s-QHQ53VrJJI9FcJsFiEmyhMpks5qjbvHgaJpZM4Jsq4- .

mrshirts commented 8 years ago

Following up from Mobley - If the data sets are that large, the biggest issue is certaintly is pymbar, not alchemical-analysis. Echoing from Mobley as well - I would definitely make sure you decorrelate your data first. The additional data doesn't really improve your estimate.

pymbar version 3.0 beta is definitely faster for larger data sets, though it's sometimes throws some errors (which is why it's still beta).

On Thu, Aug 25, 2016 at 9:26 AM, raziel81 notifications@github.com wrote:

specifically, i'm computing relatively big data sets (~40 GBs of samples) via alchemical analysis. I'm using an intel hexacore i7 with 32GB and an Nvidia 780 Ti GPU and it seems like a tremendous waste of time just to wait on a single core calculation. I won't mind making my own efforts in this matter, but I wanted to be sure that any such modifications (even just using cython to compile some of the code) will not modify potential results as far as the developers are aware (although their really is just one way to know this for sure).

On Wed, Aug 24, 2016 at 10:35 PM, Michael Shirts <notifications@github.com

wrote:

My general though is "never optimize prematurely". Are there specific applications where running the free energy analysis is a bottleneck?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/MobleyLab/alchemical-analysis/issues/91# issuecomment-242279005, or mute the thread https://github.com/notifications/unsubscribe-auth/ AOTcZhBhGOhu470LcLeBRGVHDroibraMks5qjRumgaJpZM4Jsq4- .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/MobleyLab/alchemical-analysis/issues/91#issuecomment-242428723, or mute the thread https://github.com/notifications/unsubscribe-auth/AEE31D_uOvefjRzItvQCwZfnvG1isQWyks5qjbQmgaJpZM4Jsq4- .

davidlmobley commented 8 years ago

Yes, to further clarify -- all of the (time-consuming) actual analysis here is done by pymbar. alchemical-analysis basically just processes things into numpy arrays, does bookkeeping, etc., decides what to feed into pymbar. But the intensive calculations are entirely within pymbar.

raziel81 commented 8 years ago

thank you very much for your advice Michael. Just to be sure i'm not missing out on anything - doesn't alchemical analysis decorrelate data sets automatically? At least that is the impression I got from its output at several points.

On Thu, Aug 25, 2016 at 10:59 AM, Michael Shirts notifications@github.com wrote:

Following up from Mobley - If the data sets are that large, the biggest issue is certaintly is pymbar, not alchemical-analysis. Echoing from Mobley as well - I would definitely make sure you decorrelate your data first. The additional data doesn't really improve your estimate.

pymbar version 3.0 beta is definitely faster for larger data sets, though it's sometimes throws some errors (which is why it's still beta).

On Thu, Aug 25, 2016 at 9:26 AM, raziel81 notifications@github.com wrote:

specifically, i'm computing relatively big data sets (~40 GBs of samples) via alchemical analysis. I'm using an intel hexacore i7 with 32GB and an Nvidia 780 Ti GPU and it seems like a tremendous waste of time just to wait on a single core calculation. I won't mind making my own efforts in this matter, but I wanted to be sure that any such modifications (even just using cython to compile some of the code) will not modify potential results as far as the developers are aware (although their really is just one way to know this for sure).

On Wed, Aug 24, 2016 at 10:35 PM, Michael Shirts < notifications@github.com

wrote:

My general though is "never optimize prematurely". Are there specific applications where running the free energy analysis is a bottleneck?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/MobleyLab/alchemical-analysis/issues/91# issuecomment-242279005, or mute the thread https://github.com/notifications/unsubscribe-auth/ AOTcZhBhGOhu470LcLeBRGVHDroibraMks5qjRumgaJpZM4Jsq4- .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/MobleyLab/alchemical-analysis/issues/91# issuecomment-242428723, or mute the thread https://github.com/notifications/unsubscribe-auth/AEE31D_ uOvefjRzItvQCwZfnvG1isQWyks5qjbQmgaJpZM4Jsq4-

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/MobleyLab/alchemical-analysis/issues/91#issuecomment-242463616, or mute the thread https://github.com/notifications/unsubscribe-auth/AOTcZjgDvgHOP_1-dNhn52ptPqFWI35hks5qjcn2gaJpZM4Jsq4- .

mrshirts commented 8 years ago

It does if you ask it to, so if you are seeing decorrelated output, then yes, it is already. Just wanted to make sure.

On Thu, Aug 25, 2016 at 11:15 AM, raziel81 notifications@github.com wrote:

thank you very much for your advice Michael. Just to be sure i'm not missing out on anything - doesn't alchemical analysis decorrelate data sets automatically? At least that is the impression I got from its output at several points.

On Thu, Aug 25, 2016 at 10:59 AM, Michael Shirts <notifications@github.com

wrote:

Following up from Mobley - If the data sets are that large, the biggest issue is certaintly is pymbar, not alchemical-analysis. Echoing from Mobley as well - I would definitely make sure you decorrelate your data first. The additional data doesn't really improve your estimate.

pymbar version 3.0 beta is definitely faster for larger data sets, though it's sometimes throws some errors (which is why it's still beta).

On Thu, Aug 25, 2016 at 9:26 AM, raziel81 notifications@github.com wrote:

specifically, i'm computing relatively big data sets (~40 GBs of samples) via alchemical analysis. I'm using an intel hexacore i7 with 32GB and an Nvidia 780 Ti GPU and it seems like a tremendous waste of time just to wait on a single core calculation. I won't mind making my own efforts in this matter, but I wanted to be sure that any such modifications (even just using cython to compile some of the code) will not modify potential results as far as the developers are aware (although their really is just one way to know this for sure).

On Wed, Aug 24, 2016 at 10:35 PM, Michael Shirts < notifications@github.com

wrote:

My general though is "never optimize prematurely". Are there specific applications where running the free energy analysis is a bottleneck?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/MobleyLab/alchemical-analysis/issues/91# issuecomment-242279005, or mute the thread https://github.com/notifications/unsubscribe-auth/ AOTcZhBhGOhu470LcLeBRGVHDroibraMks5qjRumgaJpZM4Jsq4- .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/MobleyLab/alchemical-analysis/issues/91# issuecomment-242428723, or mute the thread https://github.com/notifications/unsubscribe-auth/AEE31D_ uOvefjRzItvQCwZfnvG1isQWyks5qjbQmgaJpZM4Jsq4-

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/MobleyLab/alchemical-analysis/issues/91# issuecomment-242463616, or mute the thread https://github.com/notifications/unsubscribe-auth/AOTcZjgDvgHOP_1- dNhn52ptPqFWI35hks5qjcn2gaJpZM4Jsq4- .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/MobleyLab/alchemical-analysis/issues/91#issuecomment-242469370, or mute the thread https://github.com/notifications/unsubscribe-auth/AEE31PyMOkplpYV_u3pR65P4NJPTyQIRks5qjc3GgaJpZM4Jsq4- .