In the HTTP API for looper we are building, we want to capture stdout and stderr and return it to the user. Unfortunately, this capturing (using contextlib.redirect_std{out,err}) seems not to be working completely with looper. As an example, in this demo branch we run the whole main() function of cli_looper.py in the above context managers, but only a very small fraction of the actual logging output is captured - see below output.
Does one of the looper / logmuse developers have any insights why that is the case and, possibly, how it could be fixed?
Also cc @zz1874
[simeon@courgette:~/projects/uva/hello_looper]$ looper run
Looper version: 1.6.0
Command: run
Using looper config (.looper.yaml).
Using default divvy config. You may specify in env var: ['DIVCFG']
Pipestat compatible: False
## [1 of 2] sample: frog_1; pipeline: count_lines
Writing script to /home/simeon/projects/uva/hello_looper/results/submission/count_lines_frog_1.sub
Job script (n=1; 0.00Gb): /home/simeon/projects/uva/hello_looper/results/submission/count_lines_frog_1.sub
Compute node: courgette
Start time: 2024-01-23 17:49:53
/home/simeon/projects/uva/hello_looper
Number of lines: 4
## [2 of 2] sample: frog_2; pipeline: count_lines
Writing script to /home/simeon/projects/uva/hello_looper/results/submission/count_lines_frog_2.sub
Job script (n=1; 0.00Gb): /home/simeon/projects/uva/hello_looper/results/submission/count_lines_frog_2.sub
Compute node: courgette
Start time: 2024-01-23 17:49:53
/home/simeon/projects/uva/hello_looper
Number of lines: 7
Looper finished
Samples valid for job generation: 2 of 2
Commands submitted: 2 of 2
Jobs submitted: 2
Captured stderr:
Captured stdout:
/home/simeon/projects/uva/hello_looper/pipeline/count_lines.sh data/frog1_data.txt
/home/simeon/projects/uva/hello_looper/pipeline/count_lines.sh data/frog2_data.txt
In the HTTP API for
looper
we are building, we want to capturestdout
andstderr
and return it to the user. Unfortunately, this capturing (usingcontextlib.redirect_std{out,err}
) seems not to be working completely withlooper
. As an example, in this demo branch we run the wholemain()
function ofcli_looper.py
in the above context managers, but only a very small fraction of the actual logging output is captured - see below output.Does one of the
looper
/logmuse
developers have any insights why that is the case and, possibly, how it could be fixed?Also cc @zz1874