pelias / api

HTTP API for Pelias Geocoder
http://pelias.io
MIT License
219 stars 162 forks source link

Missing search result despite record being available in Elasticsearch #1669

Closed roshnet-decathlon closed 10 months ago

roshnet-decathlon commented 11 months ago

Describe the bug On searching for "Bengaluru" on Geocode.earth, we see the result Bangalore, KA, India as the topmost result.

However, a self-hosted full planet build does not return the record in search results, despite being imported in Elasticsearch.

Steps to Reproduce

  1. Search for "Bengaluru" on the Pelias compare tool
  2. There are two results, with first one being Bangalore, KA, India
  3. Searching in the self-hosted build does not include the above place in the results. a. Out of the two results in Geocode.earth results, only the second result is returned in the self-hosted version.

Expected behavior The search results in self-hosted installation and Geocode.earth should be identical, since the record can be found in Elasticsearch.

Environment (please complete the following information):

OS: Linux (Ubuntu 20)

Additional context We are running a full planet Pelias build in a VM using the Docker setup.

roshnet-decathlon commented 11 months ago

Hi Peter @missinglink, thank you for building this amazing service.

May I check if you would have some thoughts on why this could be occurring in our setup?

missinglink commented 11 months ago

Do you have this record in your build? https://pelias.github.io/compare/#/v1/place?ids=whosonfirst%3Alocality%3A102030819

Can you please run the pelias elastic stats command and post the output.

roshnet-decathlon commented 11 months ago

@missinglink thank you for your reply. We're able to find the record on visiting the corresponding endpoint in our build.

On investigating, we found issues with our Elasticsearch config -

$ pelias elastic status
000

$ pelias elastic stats
# Does not return anything, exits with a non-zero error code

$ pelias elastic wait
waiting for elasticsearch service to come up
..............................

Elasticsearch did not come up, check configuration

To debug, we're seeing logs with pelias compose logs elasticsearch and checking the lines after "level": "ERROR" -

^[[36melasticsearch_1  |^[[0m {"type": "server", "timestamp": "2023-12-10T14:02:48,466Z", "level": "ERROR", "component": "o.e.b.ElasticsearchUncaug     htExceptionHandler", "cluster.name": "pelias-dev", "node.name": "a97a4a790bd0", "message": "uncaught exception in thread [main]",
4255 ^[[36melasticsearch_1  |^[[0m "stacktrace": ["org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to obtain node      locks, tried [[/usr/share/elasticsearch/data]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without incr     easing [node.max_local_storage_nodes] (was [1])?",
4256 ^[[36melasticsearch_1  |^[[0m "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[elasticsearch-7.16.1.jar:7.16.1]",
4257 ^[[36melasticsearch_1  |^[[0m "at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:157) ~[elasticsearch-7.16.1.jar:7.16.1]",
4258 ^[[36melasticsearch_1  |^[[0m "at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:77) ~[elasticsearch-7.16.1.jar     :7.16.1]",
4259 ^[[36melasticsearch_1  |^[[0m "at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:112) ~[elasticsearch-cli-7.16.1.jar:7.16.1]",
4260 ^[[36melasticsearch_1  |^[[0m "at org.elasticsearch.cli.Command.main(Command.java:77) ~[elasticsearch-cli-7.16.1.jar:7.16.1]",
4261 ^[[36melasticsearch_1  |^[[0m "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:122) ~[elasticsearch-7.16.1.jar:7.16.1]",
4262 ^[[36melasticsearch_1  |^[[0m "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:80) ~[elasticsearch-7.16.1.jar:7.16.1]",
4263 ^[[36melasticsearch_1  |^[[0m "Caused by: java.lang.IllegalStateException: failed to obtain node locks, tried [[/usr/share/elasticsearch/data]] wit     h lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?",
4264 ^[[36melasticsearch_1  |^[[0m "at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:328) ~[elasticsearch-7.16.1.jar:7.16.1]",
4265 ^[[36melasticsearch_1  |^[[0m "at org.elasticsearch.node.Node.<init>(Node.java:427) ~[elasticsearch-7.16.1.jar:7.16.1]",
4266 ^[[36melasticsearch_1  |^[[0m "at org.elasticsearch.node.Node.<init>(Node.java:309) ~[elasticsearch-7.16.1.jar:7.16.1]",
4267 ^[[36melasticsearch_1  |^[[0m "at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:234) ~[elasticsearch-7.16.1.jar:7.16.1]",
4268 ^[[36melasticsearch_1  |^[[0m "at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:234) ~[elasticsearch-7.16.1.jar:7.16.1]",
4269 ^[[36melasticsearch_1  |^[[0m "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:434) ~[elasticsearch-7.16.1.jar:7.16.1]",
4270 ^[[36melasticsearch_1  |^[[0m "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:166) ~[elasticsearch-7.16.1.jar:7.16.1]",
4271 ^[[36melasticsearch_1  |^[[0m "... 6 more",
4272 ^[[36melasticsearch_1  |^[[0m "Caused by: java.io.IOException: failed to obtain lock on /usr/share/elasticsearch/data/nodes/0",
4273 ^[[36melasticsearch_1  |^[[0m "at org.elasticsearch.env.NodeEnvironment$NodeLock.<init>(NodeEnvironment.java:236) ~[elasticsearch-7.16.1.jar:7.16.1     ]",
4274 ^[[36melasticsearch_1  |^[[0m "at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:298) ~[elasticsearch-7.16.1.jar:7.16.1]",
4275 ^[[36melasticsearch_1  |^[[0m "at org.elasticsearch.node.Node.<init>(Node.java:427) ~[elasticsearch-7.16.1.jar:7.16.1]",
4276 ^[[36melasticsearch_1  |^[[0m "at org.elasticsearch.node.Node.<init>(Node.java:309) ~[elasticsearch-7.16.1.jar:7.16.1]",
4277 ^[[36melasticsearch_1  |^[[0m "at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:234) ~[elasticsearch-7.16.1.jar:7.16.1]",
4278 ^[[36melasticsearch_1  |^[[0m "at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:234) ~[elasticsearch-7.16.1.jar:7.16.1]",
4279 ^[[36melasticsearch_1  |^[[0m "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:434) ~[elasticsearch-7.16.1.jar:7.16.1]",
4280 ^[[36melasticsearch_1  |^[[0m "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:166) ~[elasticsearch-7.16.1.jar:7.16.1]",
4281 ^[[36melasticsearch_1  |^[[0m "... 6 more",
4282 ^[[36melasticsearch_1  |^[[0m "Caused by: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data/nodes/0/node.lock",
4283 ^[[36melasticsearch_1  |^[[0m "at sun.nio.fs.UnixException.translateToIOException(UnixException.java:90) ~[?:?]",

This looks like a permissions-related issue, which we will further debug and update in the thread.

Please feel free add any comments, would be helpful, thanks!

roshnet-decathlon commented 11 months ago

Hi @missinglink, here's the output of pelias elastic stats from our installation -

{
  "took" : 13924,
  "timed_out" : false,
  "_shards" : {
    "total" : 12,
    "successful" : 12,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 10000,
      "relation" : "gte"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "sources" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "openaddresses",
          "doc_count" : 488369074,
          "layers" : {
            "doc_count_error_upper_bound" : 0,
            "sum_other_doc_count" : 0,
            "buckets" : [
              {
                "key" : "address",
                "doc_count" : 488369074
              }
            ]
          }
        },
        {
          "key" : "openstreetmap",
          "doc_count" : 176904356,
          "layers" : {
            "doc_count_error_upper_bound" : 0,
            "sum_other_doc_count" : 0,
            "buckets" : [
              {
                "key" : "address",
                "doc_count" : 139950685
              },
              {
                "key" : "venue",
                "doc_count" : 36953671
              }
            ]
          }
        },
        {
          "key" : "geonames",
          "doc_count" : 12422425,
          "layers" : {
            "doc_count_error_upper_bound" : 0,
            "sum_other_doc_count" : 0,
            "buckets" : [
              {
                "key" : "venue",
                "doc_count" : 6857169
              },
              {
                "key" : "locality",
                "doc_count" : 5146605
              },
              {
                "key" : "neighbourhood",
                "doc_count" : 334995
              },
              {
                "key" : "county",
                "doc_count" : 45364
              },
              {
                "key" : "localadmin",
                "doc_count" : 33755
              },
              {
                "key" : "region",
                "doc_count" : 3921
              },
              {
                "key" : "macrocounty",
                "doc_count" : 320
              },
              {
                "key" : "country",
                "doc_count" : 193
              },
              {
                "key" : "macroregion",
                "doc_count" : 56
              },
              {
                "key" : "dependency",
                "doc_count" : 42
              },
              {
                "key" : "borough",
                "doc_count" : 5
              }
            ]
          }
        },
        {
          "key" : "whosonfirst",
          "doc_count" : 8300851,
          "layers" : {
            "doc_count_error_upper_bound" : 0,
            "sum_other_doc_count" : 0,
            "buckets" : [
              {
                "key" : "locality",
                "doc_count" : 4756318
              },
              {
                "key" : "postalcode",
                "doc_count" : 3060064
              },
              {
                "key" : "neighbourhood",
                "doc_count" : 297293
              },
              {
                "key" : "localadmin",
                "doc_count" : 138262
              },
              {
                "key" : "county",
                "doc_count" : 41535
              },
              {
                "key" : "region",
                "doc_count" : 4245
              },
              {
                "key" : "macrohood",
                "doc_count" : 1228
              },
              {
                "key" : "macrocounty",
                "doc_count" : 575
              },
              {
                "key" : "borough",
                "doc_count" : 450
              },
              {
                "key" : "marinearea",
                "doc_count" : 402
              },
              {
                "key" : "country",
                "doc_count" : 213
              },
              {
                "key" : "disputed",
                "doc_count" : 100
              },
              {
                "key" : "macroregion",
                "doc_count" : 100
              },
              {
                "key" : "dependency",
                "doc_count" : 40
              },
              {
                "key" : "empire",
                "doc_count" : 11
              },
              {
                "key" : "continent",
                "doc_count" : 8
              },
              {
                "key" : "ocean",
                "doc_count" : 7
              }
            ]
          }
        },
        {
          "key" : "mothership",
          "doc_count" : 3652,
          "layers" : {
            "doc_count_error_upper_bound" : 0,
            "sum_other_doc_count" : 0,
            "buckets" : [
              {
                "key" : "venue",
                "doc_count" : 3652
              }
            ]
          }
        },
        {
          "key" : "pelias",
          "doc_count" : 1,
          "layers" : {
            "doc_count_error_upper_bound" : 0,
            "sum_other_doc_count" : 0,
            "buckets" : [
              {
                "key" : "locality",
                "doc_count" : 1
              }
            ]
          }
        }
      ]
    }
  }
}
roshnet-decathlon commented 11 months ago

Also, we see that the record for Bengaluru is present in Elasticsearch - Screenshot 2023-12-06 at 11 30 16 PM

roshnet-decathlon commented 11 months ago

I'd also like to share something strange we noticed in our Pelias setup -

  1. On shutting down the Docker container for Elasticsearch and performing a search, we are still able to see API responses!
  2. However, on searching for a place (like Antarctica) that has never been searched for before, we get an empty features array
  3. On searching for Antarctica again after starting Elasticsearch, we see results as expected
  4. Shutting down the Elasticsearch container and re-searching again gives us non-empty results

It seems the queries are being cached upon search.

@missinglink, could you please help understand how to disable caching? Thank you!

missinglink commented 11 months ago

It seems the queries are being cached upon search

In the browser? If the network tab in the developer tools says the status is 304 Not Modified then it's due to the Cache-Control headers, in most browsers you can tick 'Disable Cache' in that window to tell the browser not to cache HTTP requests.

I don't believe there is any other caching, if you send a request from the command-line I'd expect it to never cache.

roshnet-decathlon commented 11 months ago

Thanks for the quick response @missinglink. However, we're referring to the raw /v1/search endpoint response (not the browser search :))

We're seeing the same behaviour from cURL as well.

roshnet-decathlon commented 11 months ago

To provide more context, here's more details on what we did -

  1. We ran docker stop pelias_elasticsearch and verified with netstat -ntlp that Elasticsearch is down (nothing using port 9200)
  2. Performed search for Texas (something we'd already searched earlier a few times) with /v1/search?text=texas -> we're getting results using cURL
  3. However, at this point we'd never searched for Antarctica, and /v1/search?text=antarctica returns empty features array
  4. We run docker start pelias_elasticsearch, and confirm with pelias elastic wait that Elasticsearch is ready on port 9200 (also confirmed wirth netstat -ntlp)
  5. Perform a search for Antarctica -> receiving results
  6. Shut down Elasticsearch with docker stop pelias_elasticsearch
  7. Repeat step (3) again -> we're getting results for Antarctica this time!
missinglink commented 11 months ago

oh, it might be that admin-only queries are being answered by Placeholder, although I forget exactly how this is configured.

can you try searching something which isn't a city/state/country, something like an address and see if you get the same result.

can you also try shutting down the placeholder service when you shut down elasticsearch and seeing what happens then?

roshnet-decathlon commented 11 months ago

Just tried this for Berliner Straße 86, Potsdam, BB, Germany along with few other entries with layer=address which we likely never searched for before.

With Elasticsearch down and all other services (including Placeholder) up, we receive non-empty data.

With Placeholder down, we get an empty response with the error response getaddrinfo ENOTFOUND placeholder.

missinglink commented 11 months ago

But is the non-empty data only administrative regions?

roshnet-decathlon commented 11 months ago

Please excuse my understanding of administrative regions, but we get this solo record in the non-empty response -

"properties": {
  "id": "101748815",
  "gid": "whosonfirst:locality:101748815",
  "layer": "locality",
  "source": "whosonfirst",
  "source_id": "101748815",
  "country_code": "DE",
  "name": "Potsdam",
  "confidence": 0.6,
  "match_type": "fallback",
  "accuracy": "centroid",
  "country": "Germany",
  "country_gid": "whosonfirst:country:85633111",
  "country_a": "DEU",
  "region": "Brandenburg",
  "region_gid": "whosonfirst:region:85682553",
  "region_a": "BB",
  "county": "Potsdam",
  "county_gid": "whosonfirst:county:102063931",
  "county_a": "PT",
  "localadmin": "Potsdam",
  "localadmin_gid": "whosonfirst:localadmin:1377694195",
  "locality": "Potsdam",
  "locality_gid": "whosonfirst:locality:101748815",
  "continent": "Europe",
  "continent_gid": "whosonfirst:continent:102191581",
  "label": "Potsdam, BB, Germany"
}
missinglink commented 10 months ago

By administrative regions I mean things like cities/regions/countries, these populated places form the place hierarchy such that a address/street/venue is parented by one or more administrative regions.

From what I can see from what you've posted the API is returning only administrative records when elasticsearch is disabled, these are provided by the placeholder docker container.

You'll notice that the results differ depending on whether elasticsearch is online or not, when online a query such as Berliner Straße 86, Potsdam, BB, Germany will return address records, but when it's offline you'll only get results for the region, in this case it's returning Potsdam.

This isn't a 'cache' per-se since the results are different depending on which backend services are online.

roshnet-decathlon commented 10 months ago

Interesting to know! Thank you for the clarification.

When searching for the same address in the Pelias compare tool , we see different name property as compared to our installation.

I think we'll proceed to re-import data into our installation and perhaps re-create the placeholder service to return updated records from Elasticsearch. Will post our findings here.

Thank you for your help!

siddharthshubhampal commented 10 months ago

Hey @missinglink,

I've repeatedly imported the India: whosonfirst dataset using the docker setup. Despite restarting the pelias-api and running 'pelias compose down' and 'pelias compose up', my local search for 'Bengaluru' yields only one (county) result: Bangalore Rural, KA, India (whosonfirst:county:890503869).

However, the same search on the Pelias compare tool returns two results - both the locality and county:

I'm puzzled as to why I'm not getting the locality result (Bangalore, KA, India - whosonfirst:locality:102030819) in my local setup. I confirmed that whosonfirst:locality:102030819 data exists in Elasticsearch, which is operational with green health status:

Here are the docker container logs for the 'Bengaluru' query:

07:30:28.987956 [wof-libpostal-server] STATUS parse 'bengaluru' 73.7µs
took: 0.417ms
parent not found! continent_id 102191569
info: [placeholder] ::ffff:172.26.0.12 - GET /parser/search?text=bengaluru&lang=eng HTTP/1.1 200 493 - 1.701 ms
info: [api] placeholder response_time=5, text=bengaluru, size=10, layers=[venue, street, country, macroregion, region, county, localadmin, locality, borough, neighbourhood, continent, empire, dependency, macrocounty, macrohood, microhood, disputed, postalcode, ocean, marinearea], private=false, focus.point.lat=45.52, focus.point.lon=-122.67, name=English, iso6391=en, iso6393=eng, via=header, defaulted=false, querySize=20, parser=libpostal, city=bengaluru, result_count=0, text_length=9, controller=placeholder
info: [api] language response_time=4, language=en, controller=language
info: [placeholder] ::ffff:172.26.0.12 - GET /parser/findbyid?ids=85632469%2C890503869%2C85672165&lang=eng HTTP/1.1 200 1048 - 0.985 ms
info: [api] ::ffff:117.235.183.138 - - [24/Dec/2023:07:30:29 +0000] "GET /v1/search?text=bengaluru HTTP/1.1" 200 1511

Here is ES status showing green:

curl -X GET "localhost:9200/_cat/indices?v"
health status index            uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   .geoip_databases aroKrZJTR-G3d1MvwzUoYQ   1   0         42            0     39.6mb         39.6mb
green  open   pelias           b9j99b_4QCe-A5Srha5ENg   6   0    3221118       323028      1.4gb          1.4gb

And the docker container statuses are as follows:

CONTAINER ID   IMAGE                         COMMAND                  CREATED        STATUS        PORTS                                                NAMES
bb0e549467cf   pelias/elasticsearch:7.16.1   "/bin/tini -- /usr/l…"   23 hours ago   Up 23 hours   127.0.0.1:9200->9200/tcp, 127.0.0.1:9300->9300/tcp   pelias_elasticsearch
f464c87aa688   pelias/api:master             "./bin/start"            23 hours ago   Up 23 hours   0.0.0.0:4000->4000/tcp                               pelias_api
e29da415d9ef   pelias/interpolation:master   "./interpolate serve…"   23 hours ago   Up 23 hours   127.0.0.1:4300->4300/tcp                             pelias_interpolation
006d1a1ed56b   pelias/placeholder:master     "./cmd/server.sh"        23 hours ago   Up 23 hours   0.0.0.0:4100->4100/tcp                               pelias_placeholder
b97a8f2cb36c   pelias/libpostal-service      "/bin/wof-libpostal-…"   23 hours ago   Up 23 hours   127.0.0.1:4400->4400/tcp                             pelias_libpostal
6bb199b4a673   pelias/pip-service:master     "./bin/start"            23 hours ago   Up 23 hours   127.0.0.1:4200->4200/tcp                             pelias_pip-service

Additionally, here are the Pelias Elasticsearch stats:

{
  "took" : 5,
  "timed_out" : false,
  "_shards" : {
    "total" : 6,
    "successful" : 6,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 10000,
      "relation" : "gte"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "sources" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "openaddresses",
          "doc_count" : 1389535,
          "layers" : {
            "doc_count_error_upper_bound" : 0,
            "sum_other_doc_count" : 0,
            "buckets" : [
              {
                "key" : "address",
                "doc_count" : 1389535
              }
            ]
          }
        },
        {
          "key" : "whosonfirst",
          "doc_count" : 1276292,
          "layers" : {
            "doc_count_error_upper_bound" : 0,
            "sum_other_doc_count" : 0,
            "buckets" : [
              {
                "key" : "locality",
                "doc_count" : 1078463
              },
              {
                "key" : "neighbourhood",
                "doc_count" : 137264
              },
              {
                "key" : "postalcode",
                "doc_count" : 32988
              },
              {
                "key" : "localadmin",
                "doc_count" : 23495
              },
              {
                "key" : "county",
                "doc_count" : 3783
              },
              {
                "key" : "macrohood",
                "doc_count" : 115
              },
              {
                "key" : "region",
                "doc_count" : 88
              },
              {
                "key" : "borough",
                "doc_count" : 87
              },
              {
                "key" : "dependency",
                "doc_count" : 7
              },
              {
                "key" : "country",
                "doc_count" : 2
              }
            ]
          }
        },
        {
          "key" : "openstreetmap",
          "doc_count" : 547377,
          "layers" : {
            "doc_count_error_upper_bound" : 0,
            "sum_other_doc_count" : 0,
            "buckets" : [
              {
                "key" : "address",
                "doc_count" : 467788
              },
              {
                "key" : "street",
                "doc_count" : 42967
              },
              {
                "key" : "venue",
                "doc_count" : 36622
              }
            ]
          }
        },
        {
          "key" : "transit",
          "doc_count" : 7911,
          "layers" : {
            "doc_count_error_upper_bound" : 0,
            "sum_other_doc_count" : 0,
            "buckets" : [
              {
                "key" : "stops",
                "doc_count" : 7672
              },
              {
                "key" : "bikeshare",
                "doc_count" : 239
              }
            ]
          }
        },
        {
          "key" : "pelias",
          "doc_count" : 3,
          "layers" : {
            "doc_count_error_upper_bound" : 0,
            "sum_other_doc_count" : 0,
            "buckets" : [
              {
                "key" : "address",
                "doc_count" : 1
              },
              {
                "key" : "example_layer",
                "doc_count" : 1
              },
              {
                "key" : "with_custom_data",
                "doc_count" : 1
              }
            ]
          }
        }
      ]
    }
  }
}

I performed an Elasticsearch query to verify the presence of whosonfirst:locality:102030819:

curl -X GET "localhost:9200/pelias/_search?pretty" -H 'Content-Type: application/json' -d'
{ "query": {
    "bool": {
      "must": [
        { "term": { "source": "whosonfirst" } },
        { "term": { "source_id": "102030819" } }
      ]
    }
  }
}'

{
  "took" : 16,
  "timed_out" : false,
  "_shards" : {
    "total" : 6,
    "successful" : 6,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 13.91156,
    "hits" : [
      {
        "_index" : "pelias",
        "_type" : "_doc",
        "_id" : "whosonfirst:locality:102030819",
        "_score" : 13.91156,
        "_source" : {
          "center_point" : {
            "lon" : 77.604785,
            "lat" : 12.970541
          },
          "parent" : {
            "country" : [
              "India"
            ],
            "country_id" : [
              "85632469"
            ],
            "country_a" : [
              "IND"
            ],
            "country_source" : [
              null
            ],
            "county" : [
              "Bangalore District"
            ],
            "county_id" : [
              "890511315"
            ],
            "county_a" : [
              null
            ],
            "county_source" : [
              null
            ],
            "locality" : [
              "Bangalore"
            ],
            "locality_id" : [
              "102030819"
            ],
            "locality_a" : [
              null
            ],
            "locality_source" : [
              null
            ],
            "region" : [
              "कर्नाटक"
            ],
            "region_id" : [
              "85672165"
            ],
            "region_a" : [
              "KA"
            ],
            "region_source" : [
              null
            ]
          },
          "bounding_box" : "{\"min_lat\":12.835277,\"max_lat\":13.143131,\"min_lon\":77.466204,\"max_lon\":77.786319}",
          "name" : {
            "default" : "Bangalore",
            "am" : "በንጋሉሩ",
            "ar" : "بانغالور",
            "as" : "বাংগালোৰ",
            "az" : "Banqalor",
            "ba" : "Бангалор",
            "be" : "Горад Бангалор",
            "bn" : "বেঙ্গালুরু",
            "br" : "Bengaluru",
            "bg" : "Бенгалуру",
            "cs" : "Bengalúru",
            "ce" : "Бангалор",
            "kw" : "Bengalour",
            "el" : "Μπανγκαλόρ",
            "eo" : "Bengaluro",
            "fa" : "بنگلور",
            "fj" : "Begaluru",
            "gu" : "બેંગ્લોર",
            "ha" : "Bengaluru",
            "he" : "בנגלור",
            "hi" : "बैंगलोर",
            "hu" : "Bengaluru",
            "hy" : "Բանդալոր",
            "id" : "Bengaluru",
            "ja" : "バンガロール",
            "kn" : "ಬೆಂಗಳೂರು",
            "ks" : "بنگلور",
            "ka" : "ბენგალურუ",
            "ky" : "Бангалор шаары",
            "ko" : [
              "방갈로르",
              "벵갈루루"
            ],
            "la" : "Bengalurus",
            "lv" : "Bengalūru",
            "lt" : "Bengalūras",
            "ml" : "ബാംഗ്ലൂര്‍",
            "mr" : "बंगळूरू",
            "mk" : "Бенгалуру",
            "mt" : "Bengaluru",
            "mn" : "Бангалор",
            "my" : "ဘန်ဂလိုမြို့",
            "ne" : "बेङ्गलोर",
            "oc" : "Bengaloro",
            "or" : "ବେଙ୍ଗାଳୁରୁ",
            "os" : "Бангалор",
            "pa" : "ਬੰਗਲੌਰ",
            "pl" : "Bengaluru",
            "ps" : "بنگلور",
            "ro" : "Bengaluru",
            "ru" : "Бангалор",
            "sa" : "बेङ्गळूरु",
            "si" : "බැංගලෝර්",
            "sk" : "Bangalúr",
            "sl" : "Bengaluru",
            "so" : "Bengaluru",
            "sr" : "Бангалор",
            "ta" : "பெங்களூரு",
            "tt" : "Бәңгәлуру",
            "te" : "బెంగళూరు",
            "tg" : "Бангалор",
            "th" : "บังคาลอร์",
            "ug" : "Ban'galor",
            "uk" : "Бенґалуру",
            "ur" : "بنگلور",
            "vi" : "Bengaluru",
            "yi" : "באנגאלאר",
            "zh" : "班加羅爾"
          },
          "addendum" : {
            "concordances" : "{\"dbp:id\":\"Bangalore\",\"fb:id\":\"en.bangalore\",\"fct:id\":\"1230b43e-8f76-11e1-848f-cfd5bf3ef515\",\"gn:id\":1277333,\"gp:id\":2295420,\"ks:id\":\"KS006BA92B3EF1483AA28C260ECE2EDCB0\",\"loc:id\":\"n79127101\",\"nyt:id\":\"N26167925545593557521\",\"qs_pg:id\":273924,\"wd:id\":\"Q1355\",\"wk:page\":\"Bangalore\"}"
          },
          "source" : "whosonfirst",
          "source_id" : "102030819",
          "layer" : "locality",
          "population" : 187
        }
      }
    ]
  }
}

This query confirmed its presence. Can you help me understand why the locality: Bangalore, KA, India (whosonfirst:locality:102030819) is not appearing in my local search results?

Thanks!

siddharthshubhampal commented 10 months ago

@missinglink can you please look into this issue^ we are blocked on this

roshnet-decathlon commented 10 months ago

Found the solution to import Placeholder data from the Placeholder blog instead of pelias prepare placeholder.

curl https://data.geocode.earth/placeholder/store.sqlite3.gz | gunzip > data/placeholder/store.sqlite3

Also, in case someone finds it helpful, exposing Elasticsearch in docker-compose.yml on 127.0.0.1 rather than 0.0.0.0 made it work.

Thank you @missinglink @siddharthshubhampal!

Will be closing this issue.

roshnet-decathlon commented 10 months ago

cc @roshnet