Remote code execution

[mpiraux]

We have IPython functions to run the experiments but no way to run them on a remote machine. IPython parallel support could be the way. Especially:

• https://ipyparallel.readthedocs.io/en/latest/tutorial/process.html#starting-a-cluster-with-ssh
• https://ipyparallel.readthedocs.io/en/latest/tutorial/direct.html

mpf should leverage ipyparallel and map roles to machine in the cluster. I'm unsure whether mpf should define the cluster itself. It will depend on the amount of boilerplate needed to get it to work. If it is too much then mpf will provide a simpler way.

[mpiraux]

So IPyParallel uses profile to configure a cluster. It's a bit cumbersome as it sits in a .ipython directory, and has a massive amounts of option. The way would be for mpf to expose a simple config file, i.e. yaml files, to describe the cluster and then generate for each experiment a temporary IPyParallel profile.

Also the profile can specify files to send, so that could be one way of sending code to the cluster.

We need also to find what info is needed to "describe the cluster".