search

whosonfirst / go-whosonfirst-tile38

Go package for working with Who's On First documents and the Tile38 datastore
BSD 3-Clause "New" or "Revised" License
2 stars 1 forks source link

Be more smarter about indexing the same ID twice #10

Open thisisaaronland opened 6 years ago

thisisaaronland commented 6 years ago

ID 1142028949 was accidentally added to the wrong repo (venue-kr) and subsequently moved to the correct repo (venue-tw) but was indexed twice in T38.

Because we use compound {WOFID}#{REPO} keys for indexing geometries T38 didn't think anything of it and the problem rippled down to the API as follows:

curl -X GET 'https://whosonfirst-api.mapzen.com?method=whosonfirst.places.getNearby&api_key=mapzen-xxxxxx&latitude=25.14284&longitude=121.45955&placetype=venue&extras=addr:full&format=json'

{
    "places": [
        {
            "wof:id": 1142028949,
            "wof:parent_id": "85924303",
            "wof:name": "Bamboo Curtain Studio",
            "wof:placetype": "venue",
            "wof:country": "TW",
            "wof:repo": "whosonfirst-data-venue-tw",
            "addr:full": "No.39, Ln. 88, Sec. 2, Zhongzheng E. Rd., Danshui Dist., New Taipei City 251, Taiwan"
        },
        {
            "wof:id": 1142028949,
            "wof:parent_id": "85924303",
            "wof:name": "Bamboo Curtain Studio",
            "wof:placetype": "venue",
            "wof:country": "TW",
            "wof:repo": "whosonfirst-data-venue-tw",
            "addr:full": "No.39, Ln. 88, Sec. 2, Zhongzheng E. Rd., Danshui Dist., New Taipei City 251, Taiwan"
        }
    ],

There is code to de-dupe WOF IDs in the API now but we should figure out how to be more smarter at this layer to prevent the problem from happening at all. Maybe a separate ID -> repo index?

thisisaaronland commented 6 years ago

See also: https://github.com/whosonfirst/go-whosonfirst-tile38/issues/5

thisisaaronland commented 6 years ago

Also: http://tile38.com/commands/del/

tile38-cli 
127.0.0.1:9851> GET whosonfirst 1142028949#whosonfirst-data-venue-kr
{"ok":true,"object":{"type":"Point","coordinates":[121.45946388889,25.142794444444]},"elapsed":"55.912µs"}
127.0.0.1:9851> GET whosonfirst 1142028949#whosonfirst-data-venue-tw
{"ok":true,"object":{"type":"Point","coordinates":[121.459464,25.142794]},"elapsed":"37.748µs"}
127.0.0.1:9851> DEL whosonfirst 1142028949#whosonfirst-data-venue-kr
{"ok":true,"elapsed":"48.121µs"}
127.0.0.1:9851> GET whosonfirst 1142028949#whosonfirst-data-venue-tw
{"ok":true,"object":{"type":"Point","coordinates":[121.459464,25.142794]},"elapsed":"46.291µs"}
127.0.0.1:9851> GET whosonfirst 1142028949#whosonfirst-data-venue-kr
(error) id not found