unic / ScalaWebTest

ScalaWebTest is a library for writing ScalaTest/Selenium based integration tests for web applications. It helps you with your basic setup and provides a new and very efficient approach to testing.
https://scalawebtest.org
Apache License 2.0
29 stars 9 forks source link

Improve Error Message for Gauge when checking Lists for an Element #101

Open thedodobird2 opened 5 years ago

thedodobird2 commented 5 years ago

When a JsonGauge checks a list of elements for a certain element using containsElementFitting, but doesn't fit any (for multiple reasons), the error message isn't very helpful. For example: The name attribute is only correct for one element in the list BUT another attribute of that 'correct' element does not match. This puts it on the same level and your gauge will just say it couldn't find a matching element. Even though there is one that is quite like it.

Proposal:

Imagine the JSON array would be

{
  "universities": [
    {
      "name": "Universität Leiden",
      "begin": 1948,
      "end": 1956
    },
    {
      "name": "Mathematisch Centrum Amsterdam",
      "begin": 1951,
      "end": 1959
    },
    {
      "name": "Technische Universiteit Eindhoven",
      "begin": 1962,
      "end": 1984
    },
    {
      "name": "University of Texas at Austin",
      "begin": 1984,
      "end": 1999
    }
  ]
}

And the following test would be used

"The universities array"  should "contain an element with the expected values" in {
    universities containsElementFitting values of
      """{
        | "name": "Technische Universiteit Eindhoven",
        | "begin": 1962,
        | "end": 2084
        | }""".stripMargin
  }

Then it would fail with

[{"name":"Universität Leiden","begin":1948,"end":1956},{"name":"Mathematisch Centrum Amsterdam","begin":1951,"end":1959},{"name":"Technische Universiteit Eindhoven","begin":1962,"end":1984},{"name":"University of Texas at Austin","begin":1984,"end":1999}] did not contain an element, which matched the gauge definition {
 "name": "Technische Universiteit Eindhoven",
 "begin": 1962,
 "end": 2084
 }

But preferably it would be

[{"name":"Universität Leiden","begin":1948,"end":1956},{"name":"Mathematisch Centrum Amsterdam","begin":1951,"end":1959},{"name":"Technische Universiteit Eindhoven","begin":1962,"end":1984},{"name":"University of Texas at Austin","begin":1984,"end":1999}] 
did not contain an element, which matched the gauge definition {
 "name": "Technische Universiteit Eindhoven",
 "begin": 1962,
 "end": 2084
 }
the closest matches where [
 {"name":"Technische Universiteit Eindhoven","begin":1962,"end":1984}
 Expected :2084 in 'end'
 Actual   :1984
]
DaniRey commented 4 years ago

This would be an improvement, unfortunately it requires a complete rewrite.

A) The objects in the array might have more then one level. A missing element on top-level more important then on a deeper level. Therefore we would have to switch from a depth-first to a breath-first search. B) As soon as there is a misfit an error is thrown, the search is aborted. Gathering errors adds complexity and leads to worse performance. On JSON arrays of reasonable size this shouldn't be an issue, nevertheless it might be one.