MarvinJWendt / testza

Full-featured test framework for Go! Assertions, fuzzing, input testing, output capturing, and much more! 🍕
MIT License
418 stars 21 forks source link

fix!: use stable key ordering for snapshots with maps #190

Closed jamestelfer closed 1 year ago

jamestelfer commented 1 year ago

Snapshots containing maps are unstable because iteration order of a map's keys is not guaranteed. This leads to test instability, where instability increases with the size of the map included in the snapshot.

Stability can be introduced by using the option of the go-spew library that sorts map keys before serialization (the SortKeys option). The SortKeys option only deals with keys that are native types, or types that implement Stringer or error, but this covers a significant proportion of the uses of map.

Breaking change: This change turns the SortKeys option on for all snapshots, potentially invalidating snapshots that already exist. In reality though, given the instability of maps in snapshots currently, it's unlikely any user of the library will be affected.

Further development

There is another option: SpewKeys. This extends the function of SortKeys to other types by first using go-spew to serialize the type to a string. In my opinion, this is likely to either be (a) something a library user might want to opt into, and (b) a less-likely scenario. For this reason, I've left that option as a potential future development.

See also

codecov[bot] commented 1 year ago

Codecov Report

Base: 87.02% // Head: 87.04% // Increases project coverage by +0.01% :tada:

Coverage data is based on head (7496539) compared to base (0854b36). Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #190 +/- ## ========================================== + Coverage 87.02% 87.04% +0.01% ========================================== Files 10 10 Lines 740 741 +1 ========================================== + Hits 644 645 +1 Misses 58 58 Partials 38 38 ``` | [Impacted Files](https://codecov.io/gh/MarvinJWendt/testza/pull/190?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Marvin+Wendt) | Coverage Δ | | |---|---|---| | [snapshot.go](https://codecov.io/gh/MarvinJWendt/testza/pull/190/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Marvin+Wendt#diff-c25hcHNob3QuZ28=) | `84.00% <100.00%> (+0.21%)` | :arrow_up: | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Marvin+Wendt). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Marvin+Wendt)

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.