How to install openmm-plumed with mpi and CUDA platform

[xlimke]

Hi!

I have problem in installing openmm-plumed with both mpi and CUDA platform.

I'd like to run a bias-exchange simulation with 14 replicas in Drude force field. So, I need to use OpenMM plugined with plumed (enable mpi). To run it faster, I'd like to run the simulation on a CUDA platform. However, the package supplied https://github.com/peastman/openmm-plumed is packed without mpi and CUDA. I wonder how can I custom the installation to carry out my work.

Thanks for any possible help in advance!

[raimis]

First of all, PLUMED doesn't run anything on GPU. It is purely CPU-only code. OpenMM-PLUMED allows OpenMM and PLUMED to interact. Including the case there OpenMM runs on a GPU, while PLUMED runs on CPU. The conda-forge packages (https://anaconda.org/conda-forge/openmm-torch) support that.

Regarding MPI, OpenMM and OpenMM-PLUMED have nothing to do. You need a PLUMED with MPI. As I know, the conda-forge packages (https://anaconda.org/conda-forge/plumed) are built without MPI. So, you needed to rebuild a compatible PLUMED version with MPI. A good starting point is the conda-forge recipes (https://github.com/conda-forge/plumed-feedstock).

[raimis]

Actually the PLUMED from conda-forge has MPI (https://anaconda.org/conda-forge/plumed/files?version=2.8.1). So, my guess you don't need to build anything. Just to configure properly.

[xlimke]

Thanks for your prompt help!

I have installed openmm-torch and plumed(enable mpi) seperately. I wonder whether I need to do something to let them interact with each other, like what openmm-plumed do?

Look forward to your reply!

[peastman]

I don't think you're going to be able to use MPI with the PLUMED plugin. They're based on different execution models. MPI wants to be at the top of the execution stack. You start by running mpirun, which launches copies of an MPI enabled application on different computers and tells them how to communicate with each other. But OpenMM doesn't use MPI. It expects you to launch a single process for a single simulation, and inside that process it links to PLUMED as a library, not an application.

[xlimke]

Thanks for your instruction!

As I know, OpenMM is able to run parallel tempering and replica exchange. But as you have said, it is not completed with mpi. I wonder whether I can interact plumed when doing parallel tempering, in able to carry out my work of parallel-tempering and well-tempered metadynamics on OpenMM(since it is one of the best platforms to run simulations of Drude force field).

[peastman]

Yes, you can still use it. PLUMED is designed to be used in a variety of ways: to compute the values of individual CVs during a simulation, to post-process a trajectory, to run metadynamics calculations, etc. The plugin for OpenMM uses it in one particular way: compute individual CVs during a simulation, which are then used as forces in the simulation. MPI isn't generally relevant in that case. Trying to parallelize a CV calculation across multiple computers would add far too much overhead.

It's still possible to use MPI at a higher level. For example, some of the sampling methods in openmmtools can use it to perform sampling across multiple computers. In that case, it would launch a separate process on each computer, and each process would internally load the PLUMED plugin and use it to compute CVs.

[xlimke]

Thanks for your help!

I'm still not very sure. Is it possible for OpenMM to run 14 metadynamics simulations in parallel temperature(from 300K to 475K)in Drude force field and have replica exchanges? Could you provide more information? Thank you!

[peastman]

Is that something people do? I don't think I've ever heard of metadynamics and replica exchange being combined in a single simulation. It's not clear to me how that would work, and how the different bias potentials from all the simulations would factor into the replica exchange process.

If you're sure you understand the theory behind it, you could probably hack something together combining the openmmtools replica exchange sampler and either PLUMED or the OpenMM metadynamics class. I've never tried to do that, so I'm not sure what problems you might run into.

[raimis]

In principle, OpenMM and PLUMED with MPI should work.

You start by running mpirun, which launches copies of an MPI enabled application on different computers and tells them how to communicate with each other

mpirun just lunches the indicated number of process and sets the MPI-specific environment variables. The process can anything, including the Python interpreter.

They're based on different execution models. MPI wants to be at the top of the execution stack.

MPI is just a library and there are no limitations about the stack. When PLUMED initializes MPI, it will read the aforementioned variables and setup communications. For OpenMM will be no difference as all the MPI stuff will be encapsulated in PLUMED.

[raimis]

Is that something people do? I don't think I've ever heard of metadynamics and replica exchange being combined in a single simulation.

Yes, the most popular is the parallel tempering metadynamics (https://pubs.acs.org/doi/10.1021/ja062463w) and there are many other variations of that. @xlimke could elaborate which one do you want to use?

Keep in mind that PLUMED has just the parallel bias metadynamics (https://www.plumed.org/doc-v2.8/user-doc/html/_p_b_m_e_t_a_d.html).

[xlimke]

Thanks for your help!

I'm doing Bias-Exchange Metadynamics of DNA system exactly(https://www.plumed.org/doc-v2.6/user-doc/html/belfast-8.html). I have carried out my work successfully in Gromacs by running: mpirun -np 14 gmx_mpi mdrun -s pro.tpr -multidir dirname -plumed plumed.dat -replex 1000

To further study, I want to see its performance in Drude force field. So I turn to OpenMM for help. But little information have I got on how to do this. I think your information has helped me shape some ideas. Thank you very much!

As a beginner, I would appreciate it a lot if you could provide me with some relevant tutorials or other guides!

[peastman]

For OpenMM will be no difference as all the MPI stuff will be encapsulated in PLUMED.

I don't think that would work. It assumes PLUMED is driving the simulation, not OpenMM. The only thing PLUMED gets used for is evaluating individual CVs for individual conformations. If you want to perform replica exchange with different replicas split across process, it will need to be implemented at a higher level.

It's likely the replica exchange code in openmmtools could be adapted to support this pretty easily. Some modification to the exchange criteria would presumably be needed, but that would hopefully not be hard.

[raimis]

@xlimke https://www.plumed.org/doc-v2.6/user-doc/html/belfast-8.html is an old tutorial, which use some hacks at the GROMACS side to do the bias-exchange MTD. These hacks aren't available at OpenMM.

As @peastman suggested, you probably can adapt the replica exchange code from openmtools (https://openmmtools.readthedocs.io/en/stable/multistate.html#replicaexchangesampler-replica-exchange-among-thermodynamic-states) for that purpose, but this requires some coding and good knowledge the involved method.

[xlimke]

I got it! Thanks for your help!