search

Matgenix / jobflow-remote

jobflow-remote is a Python package to run jobflow workflows on remote resources.
https://matgenix.github.io/jobflow-remote/
Other
25 stars 11 forks source link

Full-split : Remote Runner setup #214

Open DervinSun opened 1 week ago

DervinSun commented 1 week ago

I didn't quite find much explanation for this point on the documentations. How would I set up a remote runner on a different machine in a full-split setup?

Is there a specific runner option that I need to change?

gpetretto commented 1 week ago

Hi @DervinSun,

there is no particular option that needs to be set in the runner. It just means that at least some of the workers are of type: remote. There is also no configuration that needs to be done on the worker machine either, but the python environment should be equivalent to the one on the runner machine.

Does this answer your question?

DervinSun commented 1 week ago

So if I put in "jf runner start", it will automatically start the runner on the server?

gpetretto commented 1 week ago

I am not sure which machine you are referring to with the "server". The Runner needs to be executed only on one machine (and not more than one), that is responsible to manage the execution of the Jobs by sending them to the workers. Maybe this part of the documentation may help clarify the point: https://matgenix.github.io/jobflow-remote/user/introduction.html#working-principles jf runner start will start the runner on the machine where you execute the command. This will start a daemon locally. If you have a remote worker (e.g. a cluster) this will submit your jobs there on its own, so nothing needs to be executed manually on the worker, aside from the set up. Does this help clarifying the procedure?

DervinSun commented 1 week ago

Sorry, for not being very clear. When I said "server" I mean the machine that have the runner. So if I understand correctly, as long as I have a runner start on one machine, I do not need to run the "jf runner start" command on any other machine. It will just work? Let me give it a try, which should probably help me understand.

gpetretto commented 1 week ago

So if I understand correctly, as long as I have a runner start on one machine, I do not need to run the "jf runner start" command on any other machine.

Yes

It will just work?

As long as everything is propely configured, it should indeed work just like that 😃