Open muammar opened 4 years ago
https://github.com/muammar/slurm-magic/commit/afaa2218b4cd710044dd46cbfe55196ded11d29d represents a small hack for this. A better way would be something like what's described here:
https://stackoverflow.com/questions/375427/non-blocking-read-on-a-subprocess-pipe-in-python
I will keep working on that.
Hi @muammar, thanks for your interest. I've been wondering if anyone uses this package, because it's barely more functionality than the IPython kernel provides by default. You can background processes with the built in %%script magic. As you can see in the code you've already looked at it's really not doing anything "magic" here, just passing Slurm commands through to the command line.
Although
Popen
is being used to execute commands:https://github.com/NERSC/slurm-magic/blob/4c708cc137cb9f4bd5b44cf26837b466d9bf7b65/slurm_magic.py#L164
.communicate()
is a blocking process:https://github.com/NERSC/slurm-magic/blob/4c708cc137cb9f4bd5b44cf26837b466d9bf7b65/slurm_magic.py#L165
Is it possible to change this so that capturing
subprocess.PIPE
does not block?