goss-org / goss

Quick and Easy server testing/validation
https://goss.rocks
Apache License 2.0
5.5k stars 470 forks source link

Validate with json formatting always returns `successful: false` even for successful tests #840

Closed pahaeanx closed 9 months ago

pahaeanx commented 9 months ago

Describe the bug After upgrading from 0.3.x to 0.4.1 I noticed that all my tests were failing, or at least it looked like it as I use the json output and decide based on a tests successful field.

With 0.4.x it seems that this field is always false, even for tests that succeeded.

How To Reproduce

# file: goss.yaml
command:
  Always-succeeds:
    exec: /bin/true
    exit-status: 0
  Always-fails:
    exec: /bin/false
    exit-status: 0
goss v -f json | jq '.results[].successful'
false
false

Expected Behavior I'd expect successful tests to have their successful field to be true as was the case before 0.4.x.

Actual Behavior See above. Seems the successful field is always set to false and does not accurately reflect if a test has actually failed or not.

Environment:

aelsabbahy commented 9 months ago

Thank you for filing this bug. And awesome minimally reproducible example!

I'll investigate this sometime this week and fix, my guess is I'm not setting the field and it's defaulting to false.

aelsabbahy commented 9 months ago

v0.4.2 is released with this fix.

Thank you for reporting this, great catch!