search

shanejansen / touchstone

Touchstone is a testing framework for your services that focuses on component, end-to-end, and exploratory testing.
16 stars 2 forks source link

List (& dict) comparison tests/fixes #35

Closed ScottFreeCode closed 2 years ago

ScottFreeCode commented 2 years ago

Added explicit test cases for different types being compared in lists: single-value types (string used in practice), lists, and dicts. Test cases cover both "can lists of these be compared" (with just one element) and "can lists of these be compared out of order".

Five of the six cases (used to) fail; only the single dict case passes.

Similarly added tests for cases where the types of dicts' values are different – these were getting delegated to the comparison function for the type of the expected item even if the actual item is a different type. (It is possible that one of the tests might actually pass because the comparison method selected happens to be valid for use against the other type and return False "by accident"; fixing this should guarantee we get correct equals comparison for type mismatches, which should always return False by design.)

Then, added fixes for both – they now delegate to matches with quiet = True for the value comparison so types are handled correctly, and list comparison returns False if any item in expected does not match any item in actual (otherwise returns True if lengths are the same and all items in expected match an item in actual).

ScottFreeCode commented 2 years ago

Threw in tests for ANY in lists for good measure, because I had trouble figuring out whether my list comparison code would handle ANY correctly or if I needed to do anything else for it.

ScottFreeCode commented 2 years ago

@shanejansen (Just making sure Github pings you about this PR.)

shanejansen commented 2 years ago

@ScottFreeCode Love it! Awesome code and thank you for following existing patterns!