cea-hpc / clustershell

Scalable cluster administration Python framework — Manage node sets, node groups and execute commands on cluster nodes in parallel.
https://clustershell.readthedocs.io/
425 stars 85 forks source link

iter_buffers returns WorkerPopen rather than a string for nodelist #393

Open sunilangadi2 opened 6 years ago

sunilangadi2 commented 6 years ago

Code: result = {} for output,nodelist in task.iter_buffers():   for node in nodelist:     result[node] = str(output)

this produces output

{<ClusterShell.Worker.Popen.WorkerPopen object at 0x7f919c13ded0>: "invalid parameter dump_kernel'\nopen(dump_kernel) failed: No such file or directory"}

thiell commented 6 years ago

Hi @sunilangadi2,

For local "workers", the nodelist will actually consists in local Worker instances (eg. WorkerPopen) if a worker "key" is not provided earlier.

For example, you can set a custom key when calling task.shell() or task.run() (here the key is "local"):

from ClusterShell.Task import task_self

task = task_self()
task.run("echo foo", key="local")
for buf, keys in task.iter_buffers():
    for key in keys:
        print(key, buf.message())
$ python test.py
('local', 'foo')

For remote workers (eg. task.shell(..., nodes=...), the keys will be the node names. Hope that makes sense!

sunilangadi2 commented 6 years ago

Thanks for your solution @thiell