goss-org / goss

Quick and Easy server testing/validation
https://goss.rocks
Apache License 2.0
5.5k stars 470 forks source link

Breaking change in v0.4.0-rc.1 needs release doc fix #824

Closed gberche-orange closed 10 months ago

gberche-orange commented 11 months ago

Describe the bug

Given the following goss.yaml passing in 0.3.23

command:
  mysql-connect:
    exec: | 
       mysql -h 10.69.0.32 --database=default -u default -pommitedpassword        -e 'DROP TABLE IF EXISTS TEST; create table TEST(test_id int auto_increment, test_value int, primary key(test_id)); INSERT INTO TEST (TEST_VALUE) VALUES (1993);SELECT * FROM TEST'
    exit-status: 0
    stdout:
      - 1993 # make sure 1993 is present in the outout
    stderr: []

I would expect the test to pass in v0.4.0-rc.1 as I did not spot a related breaking change in release note. However it fails, likely related to int patterns

goss validate
.F

Failures/Skipped:

Command: mysql-connect: stdout:
Error
    HavePatterns matcher expects patterns to be a string. got: 
        <int>: 1993

Total Duration: 0.084s
Count: 2, Failed: 1, Skipped: 0

Expected Behavior

Add breaking change into https://github.com/goss-org/goss/releases/tag/v0.4.0-rc.1 such as:

Integer assertions such as the following 
\```yaml
    stdout:
      - 1993
\```

need to be expressed as string such as:

\```yaml
    stdout:
      - "1993"
\```

ps: github does not allow PRs against releases body such https://github.com/goss-org/goss/releases/tag/v0.4.0-rc.1 I see some projects also maintaining a CHANGELOG.md (e.g. https://github.com/hashicorp/terraform-provider-kubernetes/blob/main/CHANGELOG.md ) file for documenting breaking changes, therefore enabling contributions on this file.

aelsabbahy commented 11 months ago

Thank you, great catch. I totally missed this in my testing.

I tried looking briefly into seeing if I can make it backwards compatible, but unfortunately, in the new v4 logic I think once it's parsed as a float, I can't get consistent behavior when converting numeric back to string.

For example 15.0 will end up being "15" which is worse than documenting the breaking change.

I'll update the docs for a rc2 hopefully early next week.

Again, much appreciated, loving that people are testing the release candidate. Please feel free to open up issues/enhancements or general feedback on the approach of v4.

aelsabbahy commented 11 months ago

Updated the release docs:

https://github.com/goss-org/goss/releases/tag/v0.4.0-rc.1

Let me know if that is clear. I also discovered another undocumented breaking change (duplicate values in contains check) and I'll close this ticket.

Note to self: It may make sense to create a changelog.md doc.

gberche-orange commented 10 months ago

thanks @aelsabbahy, the updated release notes with breaking change section looks great !