pelias / docker

Run the Pelias geocoder in docker containers, including example projects.
MIT License
315 stars 218 forks source link

Error while running pelias prepare all #279

Closed december1990 closed 2 years ago

december1990 commented 2 years ago

I face some errors while running Pelias download all and pelias prepare all.

errors while running pelias download all :

Error: getaddrinfo ENOTFOUND biketownpdx.socialbicycles.com at extractValue (/code/pelias/transit/node_modules/sync-rpc/lib/index.js:165:19) at /code/pelias/transit/node_modules/sync-rpc/lib/index.js:175:12 at request (/code/pelias/transit/node_modules/sync-request/lib/index.js:28:15) at downloadFile (/code/pelias/transit/lib/prep_data.js:81:20) at /code/pelias/transit/lib/prep_data.js:34:17 at Array.forEach () at Object. (/code/pelias/transit/lib/prep_data.js:24:21) at Module._compile (internal/modules/cjs/loader.js:999:30) at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10) at Module.load (internal/modules/cjs/loader.js:863:32) { code: 'ENOTFOUND'

error: [whosonfirst] error downloading whosonfirst-data-admin-us-latest.db.bz2Error: Command failed: curl -sA 'pelias-whosonfirst/0.0.0-development' https://data.geocode.earth/wof/dist/sqlite/whosonfirst-data-admin-us-latest.db.bz2 | lbunzip2 > /data/whosonfirst/sqlite/whosonfirst-data-admin-us-latest.db lbunzip2: stdin: compressed data error: bad block header magic

and while running pelias prepare all the following error happens

/code/pelias/placeholder/node_modules/better-sqlite3/lib/methods/wrappers.js:5 return this[cppdb].prepare(sql, this, false); ^ SqliteError: database disk image is malformed at Database.prepare (/code/pelias/placeholder/node_modules/better-sqlite3/lib/methods/wrappers.js:5:21) at new SQLiteStream (/code/pelias/placeholder/node_modules/pelias-whosonfirst/src/components/sqliteStream.js:10:31) at /code/pelias/placeholder/cmd/wof_extract_sqlite.js:52:12 at CombinedStream._realGetNext (/code/pelias/placeholder/node_modules/combined-stream/lib/combined_stream.js:104:3) at CombinedStream._getNext (/code/pelias/placeholder/node_modules/combined-stream/lib/combined_stream.js:82:12) at CombinedStream.resume (/code/pelias/placeholder/node_modules/combined-stream/lib/combined_stream.js:154:10) at CombinedStream.pipe (/code/pelias/placeholder/node_modules/combined-stream/lib/combined_stream.js:66:8) at Object. (/code/pelias/placeholder/cmd/wof_extract_sqlite.js:62:4) at Module._compile (internal/modules/cjs/loader.js:999:30) at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)

missinglink commented 2 years ago

The biketownpdx.socialbicycles.com domain seems to no longer be available, we dont control that data so we should probably delete the config.

[edit] fixed in https://github.com/pelias/docker/commit/ce5b6b9e9a80c3eb9404c7c1d0324ca75030f1d7

The data.geocode.earth content is up and available, if you click the link you pasted above it should download in your browser just fine?

I suspect there is either an intermittent issue with the CDN which can be resolved by trying again or a problem with your server environment/network.

Lastly worth noting that we have new abuse protection measures in place, if an IP address is downloading the same file hundreds of times a day it will end up on a blocklist, it's rare but if you think that might be the case let me know your IP and I can check.

december1990 commented 2 years ago

what about the second error ? return this[cppdb].prepare(sql, this, false); SqliteError: database disk image is malformed that happens during pelias prepare all

missinglink commented 2 years ago

If the .db.bz file (a SQLite database) can't be downloaded and decompressed then you'll see an error when trying to load that database.

december1990 commented 2 years ago

The problem seemed to be an error downloading files and after some retries it worked. thanks