search

markovmodel / adaptivemd

A python framework to run adaptive Markov state model (MSM) simulation on HPC resources
GNU Lesser General Public License v2.1
18 stars 7 forks source link

Create a RP worker #13

Open jhprinz opened 7 years ago

jhprinz commented 7 years ago

I think that would be a good addition. Currently you submit tasks to the queue and the stand-alone worker grab jobs to complete.

The RP implementation with a scheduler requires a user to explicitely place jobs in the correct order in the RP queue and they will be executed. I would like to copy the same single-worker mechanism to auto submit jobs to an RP queue.

That means you can just add all RP clusters to the execution as well. Example

# submit some trajectory jobs
project.queue(project.new_trajectory(pdb_file, 100) for f in range(1000))

then you would start a RP job worker that will create a session for some time grab some tasks to be executed.

adaptiverpworker -d {...} --cores=100 --resource=fub.allegro --walltime=02:00:00 project_name

this can run whereever it has access to the cluster and to the DB. Options should be similar to the adativeworker