pelias / docker

Run the Pelias geocoder in docker containers, including example projects.
MIT License
314 stars 217 forks source link

Full Planet Import only takes 300GB in Elastic Search #349

Open gaeunfalo16 opened 4 days ago

gaeunfalo16 commented 4 days ago

Hi All!

I've tried import full planet project and have some questions

Acoording to pelias document, full planet builds weigh in at over 600million documents and require aoubt 450GB.

I cannot convince whether importing data is success or not.

Here is my importing log and pelias system env below... I saw "No datapath configured, nothing to do." at last and importing was finished. Does it mean that I configured wrong datapath?

info: [admin-lookup:worker] region worker process exiting, stats: {"calls":722842,"hits":433148,"misses":289694} info: [admin-lookup:worker] country worker process exiting, stats: {"calls":284582,"hits":73725,"misses":210857} info: [admin-lookup:worker] ocean worker process exiting, stats: {"calls":54997,"hits":33706,"misses":21291} info: [admin-lookup:worker] locality worker process exiting, stats: {"calls":12549816,"hits":2801773,"misses":9748043} info: [dbclient-geonames] paused=false, transient=0, current_length=0, indexed=12681984, batch_ok=25364, venue=7066398, locality=5191814, region=3932, batch_retries=0, failed_records=0, neighbourhood=339824, localadmin=32575, country=193, county=46825, persec=7448.4, dependency=42, macroregion=56, macrocounty=320, borough=5 info: [dbclient-geonames] paused=false, transient=0, current_length=0, indexed=12681984, batch_ok=25364, venue=7066398, locality=5191814, region=3932, batch_retries=0, failed_records=0, neighbourhood=339824, localadmin=32575, country=193, county=46825, persec=7448.4, dependency=42, macroregion=56, macrocounty=320, borough=5 time="2024-06-27T14:15:22+09:00" level=warning msg="/home1/irteam/docker/projects/planet/docker-compose.yml: version is obsolete" warn: [transit] 'pelias.json' config lacks a transit object entry. Transit importer quitting after taking no action time="2024-06-27T14:15:23+09:00" level=warning msg="/home1/irteam/docker/projects/planet/docker-compose.yml: version is obsolete" info: [csv-importer] No datapath configured, nothing to do