pelias / fuzzy-tester

A fuzzy testing library for geocoding
https://pelias.io
4 stars 6 forks source link

assert top two results have the same properties #73

Open missinglink opened 8 years ago

missinglink commented 8 years ago

I am trying to assert that two sections of a road are returned as the top two results, it seems like the current assertion schema doesn't support this sort of thing?

{
      "id": 2,
      "status": "pass",
      "user": "missinglink",
      "description": "road segement divided in the middle by a park",
      "notes": [
        "there should be two records, returned as result 1 & 2",
        "the testing framework doesn't support this type of test"
      ],
      "in": {
        "sources": "osm",
        "layers": "street",
        "text": "grolmanstr, berlin"
      },
      "expected": {
        "coordinates": [ 13.324042, 52.503915 ],
        "properties": [
          {
            "country_a": "DEU",
            "name": "Grolmanstraße",
            "street": "Grolmanstraße",
            "layer": "street"
          },{
            "country_a": "DEU",
            "name": "Grolmanstraße",
            "street": "Grolmanstraße",
            "layer": "street"
          }
        ]
      }
    }

adding additional duplicate blocks has no effect on the pass/fail of the test

orangejulius commented 8 years ago
{
      "id": 2,
      "status": "pass",
      "user": "missinglink",
      "description": "road segement divided in the middle by a park",
      "notes": [
        "there should be two records, returned as result 1 & 2",
        "the testing framework doesn't support this type of test"
      ],
      "in": {
        "sources": "osm",
        "layers": "street",
        "text": "grolmanstr, berlin"
      },
      "expected": {
        "priorityThresh": 2,
        "coordinates": [ 13.324042, 52.503915 ],
        "properties": [
          {
            "country_a": "DEU",
            "name": "Grolmanstraße",
            "street": "Grolmanstraße",
            "layer": "street"
          },{
            "country_a": "DEU",
            "name": "Grolmanstraße",
            "street": "Grolmanstraße",
            "layer": "street"
          }
        ]
      }
    }

Does that work? Note the priorityThresh added in the expected block.

missinglink commented 8 years ago

I tried that and it didn't work unfortunately, I'm guessing that it's not disregarding records that have already been matched and so both clauses match on the first document.

orangejulius commented 8 years ago

Ohh right, because the two property blocks are exactly the same. Yeah, right now that isn't handled properly. It would be a bit tricky to support, but we should support it some day.