Mastercard / flow

Testing framework
Apache License 2.0
18 stars 16 forks source link

Show raw data for unparseable actuals #882

Open therealryan opened 1 month ago

therealryan commented 1 month ago

When the system under test does something unexpected (e.g.: returning something that isn't json when we expected json) then the test will fail because we can't parse the actual data into the expected message type for a masked and formatted comparison. This behaviour is fine and expected.

In such circumstance the report is a bit unhelpful - by default the report will show the formatted view of the message data, but as we don't have that for the unexpected message content it shows a "No Human data available!" message instead. While that's technically correct it's easy for the viewer to get the impression that we don't have any data at all, which makes debugging much more difficult. In truth we do have the data - if you click to the "Content bytes as text" or "Content bytes" views then you can see the unexpected message content.

In cases where we don't have the formatted message but we do have the raw bytes, then we should show the raw data (with a suitable caveat message, something along the lines of "Actual data not parseable, displaying raw content") instead of just throwing up the "No Human data available!" error.