Closed gberche-orange closed 1 year ago
Thank you for filing this. There seem to be two parts:
The bug has to do with "/^$/"
, you're right that is odd behavior. I know why it's happening, and it's not intuitive for the end user. The workaround suggested in the linked issue will solve it, but it isn't evident:
command:
# the sleep command produces no stdout or stderr
sleep_test:
exec: |
>&2 echo "hi"
exit-status: 0
stderr: [ "!/./" ]
timeout: 10000
See my comment here for more info: https://github.com/goss-org/goss/issues/145#issuecomment-245045346
Essentially, the check is if any line is empty.. but there are no lines, so it fails to find an empty line. You can test this behavior with echo ""
vs echo -n ""
, the former will pass [ "/^$/" ]
the latter won't.
My memory is a bit hazy, but I feel like I explored making a special case for empty output, but it caused a breaking change somewhere.. but I could be wrong. I'll re-evaluate this with fresh eyes and see if it makes sense to special case empty string.
For the feature request, the upcoming goss v4 #814 provides a way to handle this easier:
command:
sleep_test:
exec: |
>&2 echo hi
exit-status: 0
stderr: ""
$ goss v
.F
Failures/Skipped:
Command: sleep_test: stderr:
Expected
"hi\n"
to equal
""
Total Duration: 0.001s
Count: 2, Failed: 1, Skipped: 0
It will also handle newlines correctly, for example:
$ cat goss.yaml
command:
sleep_test:
exec: |
>&2 echo ""
exit-status: 0
stderr: ""
$ goss v
.F
Failures/Skipped:
Command: sleep_test: stderr:
Expected
"\n"
to equal
""
Total Duration: 0.002s
Count: 2, Failed: 1, Skipped: 0
The only way the above will pass is with echo -n ""
I'll have to reflect a bit about defaults when doing a goss add
. It probably won't be in the initial v4 release.. but might be added in a future release.
Would love your thoughts on the above..
Thanks a lot @aelsabbahy for your prompt and detailed answer !
Bug The workaround suggested in the linked issue will solve it
Thanks for the workaround which properly detects empty strings, with the following known limitations
command:
"echo my_stdout > /dev/stdout; echo my_stderr > /dev/stderr":
exit-status: 0
stdout: [ "!/./" ]
stderr: [ "!/./" ]
timeout: 10000
$ goss validate
.FF
Failures/Skipped:
Command: echo stdout > /dev/stdout; echo stderr > /dev/stderr: stdout: patterns not found: [!/./]
Command: echo stdout > /dev/stdout; echo stderr > /dev/stderr: stderr: patterns not found: [!/./]
Total Duration: 0.002s
Count: 3, Failed: 2, Skipped: 0
feature request For the feature request, the upcoming goss v4 https://github.com/goss-org/goss/pull/814 provides a way to handle this easier
This looks great to me, fully satisfying my needs !
I'll have to reflect a bit about defaults when doing a goss add. It probably won't be in the initial v4 release.. but might be added in a future release.
With the v4 behavior for "" matcher, the current default behavior (i.e. adding "" matchers in goss add command
when observing empty strings) seems a good sensible default to me.
I'm looking forward to using goss v4 !
Forgot to mention, I released a release candidate v0.4.0-rc1 a few days (weeks?) ago.
Would love your feedback on it to make sure it correctly covers this need.
v0.4.2 is released with empty strings when there's no output.
Feel free to re-open an issue if this is still a problem. Thank you for reporting!
Describe the feature:
As a goss user, In order to detect non expected stderr/stdout messages in command output I need the
goss add command
to set assertions when observed stdout/stderr are emptyFor instance, if a binary is missing from the test environment and stderr includes "command not found", I'd need the stderr assert to fail
Describe the solution you'd like
A way to assert empty stdout/stderr similar to https://github.com/goss-org/goss/issues/184
And this syntax should be used by default in goss.yaml when the
goss add command
observes empty stdout/stderrDescribe alternatives you've considered
I tried using rexexp to check empty strings such as:
however goss isn't yet failing this assertion: