samba-in-kubernetes / sit-environment

SIT (Samba Integration Testing) Framework
GNU General Public License v3.0
1 stars 7 forks source link

test.sit-test-cases: Redirect test output to a file #67

Closed anoopcs9 closed 11 months ago

anoopcs9 commented 11 months ago

Instead of cluttering the jenkins job console output with detailed test results we could redirect everything to a separate file and make it available as artifacts for each job. Simplest way to achieve this redirection is to make use of &>(see section 3.6.4 bash manual) to include both STDOUT and STDERR. Location of such an output file is chosen to be under /var/log so that it also gets collected as part of statedump process.

spuiuk commented 11 months ago

At the moment you only capture the stdout. Maybe we should capture stderr as well with 2>&1?

anoopcs9 commented 11 months ago

At the moment you only capture the stdout.

Correct.

Maybe we should capture stderr as well with 2>&1?

But what are we missing? Everything we used to see previously should be redirected to the mentioned file. If there's something that we specifically write to stderr and not stdout, we are already(currently) missing those lines. See test.out.txt as an example for a failure case.

anoopcs9 commented 11 months ago

Maybe we should capture stderr as well with 2>&1?

But what are we missing? Everything we used to see previously should be redirected to the mentioned file. If there's something that we specifically write to stderr and not stdout, we are already(currently) missing those lines. See test.out.txt as an example for a failure case.

Here's how the console output will look like:

TASK [test.sit-test-cases : Run tests] *** Wednesday 22 November 2023 05:58:01 +0000 (0:00:00.058) 0:00:05.373 *** fatal: [client0]: FAILED! => {"changed": true, "cmd": "make test > /var/log/test.out", "delta": "0:12:21.031685", "end": "2023-11-22 06:10:22.800463", "msg": "non-zero return code", "rc": 2, "start": "2023-11-22 05:58:01.768778", "stderr": "make: [Makefile:15: test] Error 1", "stderr_lines": ["make: *** [Makefile:15: test] Error 1"], "stdout": "", "stdout_lines": []}

make: *** [Makefile:15: test] Error 1 is the one that might get missed from test output file. If required I can include that as well.

anoopcs9 commented 11 months ago

ACK

This is fine for now. If we want to consider capturing STDERR later, we can revisit this.

I've then decided to include STDERR too which seems to be a small addition.

the changes look good to me, but I generally prefer to have a bit more of a justification and explanation of the changes in the commit message and in the PR description. I. E.:

Thanks, let me try to include more details with the next version.

anoopcs9 commented 11 months ago

/retest centos-ci/cephfs

spuiuk commented 11 months ago

/test centos-ci/cephfs

spuiuk commented 11 months ago

/retest centos-ci/cephfs