Jupyter-Kale / kale

Jupyter Interactive Workflows for High Performance Computing
Other
15 stars 3 forks source link

PythonFunctionTasks via Fireworks #51

Open OliverEvans96 opened 6 years ago

OliverEvans96 commented 6 years ago

Whereas Parsl can launch a Python function given the function itself, Fireworks uses the name of the function, which it serializes and imports in the environment that it's actually run.

This is an issue for functions defined interactively in an IPython session, since they aren't defined in a .py file that can be imported remotely.

We should perhaps serialize the text of the function ourselves, create a .py file, and ship it to fireworks. We should look into how Parsl and IPyParallel handle this.