search

lordmauve / chopsticks

Chopsticks is an orchestration library: it lets you execute Python code on remote hosts over SSH.
https://chopsticks.readthedocs.io/
Apache License 2.0
158 stars 16 forks source link

Allow using Chopsticks from Jupyter Notebook #17

Closed lordmauve closed 7 years ago

lordmauve commented 7 years ago

Would be interesting to make Chopsticks available in Jupyter notebooks - ie. the ability to create a function without a notebook cell and call it on a remote host.

This would require serialising the function code, which might require a new serialisation method.

lordmauve commented 7 years ago

Functions defined in a notebook are defined within the __main__ module. inspect.getsource() can find the source code for a function defined in __main__, meaning it could be serialised as source code. It would also be possible to inspect the AST or bytecode for LOAD_GLOBAL operations, allowing the call to close over globals that are needed to execute the function (in particular, imported modules).

Special-casing the behaviour of callables defined in __main__ to work in this way would both make Chopsticks work in notebooks, as well as removing some of the complications of how to define single-file Chopsticks scripts. It would, however, be a backwards-incompatible change.

lordmauve commented 7 years ago

I have an implementation of this idea working in ed9179c38ed3d43db52862b6f0fc663a0cfa54a4.