search

pelias / docker

Run the Pelias geocoder in docker containers, including example projects.
MIT License
332 stars 224 forks source link

pelias import all hits missing datafile and crashes #54

Open jeremy-rutman opened 5 years ago

jeremy-rutman commented 5 years ago

after a day of wrangling i thought i was finally home free - all steps in the docker example up till and including download all, prepare all had run successfully, but now: 

jeremy@jeremy$ pelias import  all
info: [whosonfirst] Loading whosonfirst-data-ocean-latest.csv records from /data/whosonfirst/meta
events.js:167
      throw er; // Unhandled 'error' event
      ^

Error: ENOENT: no such file or directory, open '/data/whosonfirst/meta/whosonfirst-data-ocean-latest.csv'
Emitted 'error' event at:
    at lazyFs.open (internal/fs/streams.js:115:12)
    at FSReqWrap.oncomplete (fs.js:141:20)

should a missing data file really trigger a crash? anyway perhaps the problem is what appears to be an incorrect absolute path? data is a subdir of my working dir, so /data doesn't exist except maybe in the docker filesystem.

My DATA_DIR is /home/jeremy/pelias_dock/docker/projects/portland-metro/data , the dir exists and is writable. Maybe the use of /data got hardcoded somewhere? I see its used in the shell script example...

jeremy-rutman commented 5 years ago

a second round of 

pelias download all
pelias prepare all
pelias import all

seems to have taken care of it. I may have run out of disk space, the readme could estimate required space at 20gb (mainly wof, it seems - i guess that can't be downloaded piecemeal )

jeremy-rutman commented 5 years ago

I again ran into the same type error, 

info: [whosonfirst] Loading whosonfirst-data-postalcode-by-latest.csv records from /data/whosonfirst/meta
info: [whosonfirst] Loading whosonfirst-data-postalcode-ca-latest.csv records from /data/whosonfirst/meta
events.js:167
      throw er; // Unhandled 'error' event
      ^
Error: ENOENT: no such file or directory, open '/data/whosonfirst/meta/whosonfirst-data-postalcode-ca-latest.csv'
Emitted 'error' event at:
    at lazyFs.open (internal/fs/streams.js:115:12)
    at FSReqWrap.oncomplete (fs.js:141:20)
deploy@server01:/mnt/docker/projects/planet$

This error is more serious when doing planet as then youŕe looking potentially at a week down the drain. I would try to tackle it myself if I can get some pointers