search

pelias / docker

Run the Pelias geocoder in docker containers, including example projects.
MIT License
338 stars 226 forks source link

How to run Pelias geocoder in docker with my own .pbf file data without external data #352

Open bmyashaswi opened 3 months ago

bmyashaswi commented 3 months ago

Describe the bug I am trying to run Pelias geocoder in Docker using my own .pbf file data. I want to do this without using any external data sources. However, I am encountering issues with the setup.

Steps to Reproduce

Prepare a Docker environment with Pelias. Use a local .pbf file for data import. Configure Pelias to use this .pbf file. Start Pelias using Docker and attempt to import the .pbf file. Expected behavior Pelias should successfully import and index the data from my .pbf file and be able to handle geocoding requests using this data.

References

missinglink commented 3 months ago

This should be possible, you might have an issue accessing the pbf file in the container.

Please post your pelias.json and docker-compose.yml

bmyashaswi commented 3 months ago

Hi, Here is my pelias.json file { "esclient": { "hosts": [{ "host": "localhost", "port": 9200 }] }, "imports": { "openstreetmap": { "datapath": "/media/tech/New Volume/Yashaswi/pelias/pelias_bindu_lite_v1.5.pbf", "leveldbpath": "/media/tech/New Volume/Yashaswi/pelias/data/osm/leveldb", "adminLookup": true, "missingFilesAreFatal": false } }, "schema": { "indexName": "pelias" }, "api": { "services": { "placeholder": { "url": "http://placeholder:4100" }, "pip": { "url": "http://pip:4200" }, "interpolation": { "url": "http://interpolation:4300" } } } }

and here is my docker-compose.yml file version: '2.1' services: elasticsearch: image: pelias/elasticsearch:7.10.2 environment:

can you tell me the folder structure and procedure to run pelias geocoder in docker without external data.

missinglink commented 3 months ago

This mapping - /media/tech/yashaswi_pelias:/data says that the directory on the left will be known as the directory on the right within the container.

So you can leave your docker-comose.yml as it is but change the paths in the pelias.json to point to /data eg:

Change this:

"datapath": "/media/tech/New Volume/Yashaswi/pelias/pelias_bindu_lite_v1.5.pbf" to:

"datapath": "/data/pelias_bindu_lite_v1.5.pbf" and make sure the file is located at /media/tech/yashaswi_pelias/pelias_bindu_lite_v1.5.pbf on your local machine.

bmyashaswi commented 3 months ago

I did changes after that I am getting like this error

pelias-api_1 | Elasticsearch WARNING: 2024-07-25T04:38:05Z pelias-api_1 | Unable to revive connection: http://localhost:9200/ pelias-api_1 | pelias-api_1 | Elasticsearch WARNING: 2024-07-25T04:38:05Z pelias-api_1 | No living connections pelias-api_1 | pelias-api_1 | 2024-07-25T04:38:05.994Z - error: [api:type_mapping_discovery] message=No Living connections, stack=Error: No Living connections pelias-api_1 | at sendReqWithConnection (/code/pelias/api/node_modules/elasticsearch/src/lib/transport.js:266:15) pelias-api_1 | at next (/code/pelias/api/node_modules/elasticsearch/src/lib/connection_pool.js:243:7) pelias-api_1 | at processTicksAndRejections (node:internal/process/task_queues:78:11) pelias-api_1 | 2024-07-25T04:38:05.997Z - info: [api] pelias is now running on http://:::3100 pelias-api_1 | 2024-07-25T04:40:12.050Z - info: [api] Pelias API shutting down pelias-api_1 | 2024-07-25T04:40:23.548Z - warn: [pip] pip service disabled pelias-api_1 | 2024-07-25T04:40:23.549Z - warn: [placeholder] placeholder service disabled pelias-api_1 | 2024-07-25T04:40:23.549Z - warn: [language] language service disabled pelias-api_1 | 2024-07-25T04:40:23.549Z - warn: [interpolation] interpolation service disabled pelias-api_1 | 2024-07-25T04:40:23.549Z - warn: [libpostal] libpostal service disabled pelias-api_1 | 2024-07-25T04:40:23.549Z - warn: [libpostal] libpostal service disabled pelias-api_1 | Elasticsearch ERROR: 2024-07-25T04:40:23Z pelias-api_1 | Error: Request error, retrying pelias-api_1 | POST http://localhost:9200/_search?request_cache=true&timeout=10s => connect ECONNREFUSED 127.0.0.1:9200 pelias-api_1 | at Log.error (/code/pelias/api/node_modules/elasticsearch/src/lib/log.js:239:56) pelias-api_1 | at checkRespForFailure (/code/pelias/api/node_modules/elasticsearch/src/lib/transport.js:298:18) pelias-api_1 | at HttpConnector. (/code/pelias/api/node_modules/elasticsearch/src/lib/connectors/http.js:171:7) pelias-api_1 | at ClientRequest.wrapper (/code/pelias/api/node_modules/lodash/lodash.js:4991:19) pelias-api_1 | at ClientRequest.emit (node:events:513:28) pelias-api_1 | at Socket.socketErrorListener (node:_http_client:494:9) pelias-api_1 | at Socket.emit (node:events:513:28) pelias-api_1 | at emitErrorNT (node:internal/streams/destroy:157:8) pelias-api_1 | at emitErrorCloseNT (node:internal/streams/destroy:122:3) pelias-api_1 | at processTicksAndRejections (node:internal/process/task_queues:83:21) pelias-api_1 | pelias-api_1 | Elasticsearch WARNING: 2024-07-25T04:40:23Z pelias-api_1 | Unable to revive connection: http://localhost:9200/ pelias-api_1 | pelias-api_1 | Elasticsearch WARNING: 2024-07-25T04:40:23Z pelias-api_1 | No living connections pelias-api_1 | pelias-api_1 | 2024-07-25T04:40:23.571Z - error: [api:type_mapping_discovery] message=No Living connections, stack=Error: No Living connections pelias-api_1 | at sendReqWithConnection (/code/pelias/api/node_modules/elasticsearch/src/lib/transport.js:266:15) pelias-api_1 | at next (/code/pelias/api/node_modules/elasticsearch/src/lib/connection_pool.js:243:7) pelias-api_1 | at processTicksAndRejections (node:internal/process/task_queues:78:11) pelias-api_1 | 2024-07-25T04:40:23.573Z - info: [api] pelias is now running on http://:::3100 root@tech-HP-280-Pro-G6-Microtower-PC:/media/tech/New Volume/Yashaswi/pelias# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 051573476fee pelias/api:latest "./bin/start" 4 minutes ago Up 2 minutes 0.0.0.0:3100->3100/tcp, :::3100->3100/tcp pelias_pelias-api_1 a0a052f18b70 docker.elastic.co/elasticsearch/elasticsearch:7.15.0 "/bin/tini -- /usr/l…" 17 minutes ago Up 17 minutes 0.0.0.0:9200->9200/tcp, :::9200->9200/tcp, 9300/tcp pelias_elasticsearch_1 root@tech-HP-280-Pro-G6-Microtower-PC:/media/tech/New Volume/Yashaswi/pelias# docker ps ls "docker ps" accepts no arguments. See 'docker ps --help'.

Usage: docker ps [OPTIONS]

List containers root@tech-HP-280-Pro-G6-Microtower-PC:/media/tech/New Volume/Yashaswi/pelias# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 051573476fee pelias/api:latest "./bin/start" 4 minutes ago Up 2 minutes 0.0.0.0:3100->3100/tcp, :::3100->3100/tcp pelias_pelias-api_1 a0a052f18b70 docker.elastic.co/elasticsearch/elasticsearch:7.15.0 "/bin/tini -- /usr/l…" 18 minutes ago Up 18 minutes 0.0.0.0:9200->9200/tcp, :::9200->9200/tcp, 9300/tcp pelias_elasticsearch_1 root@tech-HP-280-Pro-G6-Microtower-PC:/media/tech/New Volume/Yashaswi/pelias# ^C

missinglink commented 3 months ago

This is a huge blob of text, please format your comments so that they are legible.

You've only posted part of the docker-compose.yml file above, please post all of it.