search

pelias / docker

Run the Pelias geocoder in docker containers, including example projects.
MIT License
314 stars 217 forks source link

This test is not failing (maybe it was previously?) #332

Closed michaelkirk closed 6 months ago

michaelkirk commented 6 months ago

Note its sibling test was marked as passing in 627ddaba21d0163036570ee0576cf45c9c1d5ce6

I imagine it's pretty challenging to keep all the integration tests up to date, given the churn/fragility of relying on externally edited datasets like openstreetmap and openaddreses.

My understanding, based on #204, is that we don't necessarily expect all the tests in this repository to be accurate, and some are aspirational, but we do want the "full planet" and portland-metro in particular to be accurate.

The "full planet" tests are presumably important for their completeness while portland-metro is presumably important because it serves as a quick-but-complete example.

Testing

I ran the instructions in the readme to import the portland-metro and run the test suite.

before

/v1/search venues
  ✔ improvement [1] "/v1/search?text=pdx"
  ✔ [2] "/v1/search?text=pdx airport"

Aggregate test results
Pass: 402
Improvements: 1
Expected Failures: 73
Placeholders: 0
Regressions: 0
Total tests: 476
Took 5174ms
Test success rate 100%

0 regressions detected. All good.

after

/v1/search venues
  ✔ [1] "/v1/search?text=pdx"
  ✔ [2] "/v1/search?text=pdx airport"

Aggregate test results
Pass: 403
Improvements: 0
Expected Failures: 73
Placeholders: 0
Regressions: 0
Total tests: 476
Took 5080ms
Test success rate 100%
missinglink commented 6 months ago

Great thanks 🙏

You're right in saying that the test results can change but it's important to keep an eye on them and make sure they're moving in the right direction, merging this should alert us if a regression is introduced to the PDX test in the future.