Can't import pelias docker for Quebec

[PhanSon95]

I have problem with import pelias address for Quebec city This is my error :

devops@pelias:~/docker/projects/quebec-city$ pelias import all
DATA_DIR is already set to './data' - this may cause the DATA_DIR specified in the .env to be ignored
/code/pelias/whosonfirst/import.js:18
    throw new Error(err.message);
    ^

Error
    at /code/pelias/whosonfirst/import.js:18:11
    at getDBList (/code/pelias/whosonfirst/src/bundleList.js:54:12)
    at Object.getList [as generateBundleList] (/code/pelias/whosonfirst/src/bundleList.js:61:12)
    at Object.<anonymous> (/code/pelias/whosonfirst/import.js:15:9)
    at Module._compile (node:internal/modules/cjs/loader:1198:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1252:10)
    at Module.load (node:internal/modules/cjs/loader:1076:32)
    at Function.Module._load (node:internal/modules/cjs/loader:911:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
    at node:internal/main/run_main_module:22:47

Steps to Reproduce This is my Quebec project struct (I copy similar new-york-city project): This is my config

//pelias.json
{
  "logger": {
    "level": "info",
    "timestamp": false
  },
  "esclient": {
    "apiVersion": "7.5",
    "hosts": [
      { "host": "elasticsearch" }
    ]
  },
  "elasticsearch": {
    "settings": {
      "index": {
        "refresh_interval": "10s",
        "number_of_replicas": "0",
        "number_of_shards": "1"
      }
    }
  },
  "acceptance-tests": {
    "endpoints": {
      "docker": "http://api:4000/v1/"
    }
  },
  "api": {
    "services": {
      "placeholder": { "url": "http://placeholder:4100" },
      "pip": { "url": "http://pip:4200" },
      "interpolation": { "url": "http://interpolation:4300" },
      "libpostal": { "url": "http://libpostal:4400" }
    }
  },
  "imports": {
    "adminLookup": {
      "enabled": true
    },
    "blacklist": {
      "files": []
    },
    "geonames": {
      "datapath": "/data/geonames",
      "countryCode": "CA"
    },
    "openstreetmap": {
      "leveldbpath": "/tmp",
      "datapath": "/data/openstreetmap",
      "import": [{
        "filename": "quebec-latest.osm.pbf"
      }]
    },
    "openaddresses": {
      "datapath": "/data/openaddresses",
      "files": [
        "ca/qc/quebec_city-addresses-city.geojson",
        "ca/qc/montreal-addresses-city.geojson"
      ]
    },
    "polyline": {
      "datapath": "/data/polylines",
      "files": [ "extract.0sv" ]
    },
    "whosonfirst": {
      "datapath": "/data/whosonfirst",
      "importPostalcodes": true,
      "countryCode": "CA",
      "importPlace": [
        "136251273",
        "890458661",
        "101736545"
      ]
    },
    "interpolation": {
      "download": {
        "tiger": {
          "datapath": "/data/tiger"
        }
      }
    }
  }
}

//.env
COMPOSE_PROJECT_NAME=quebec
DATA_DIR=./data

I also download some data:

• quebec openaddresses
• quebec openstreetmap : http://download.geofabrik.de

After that, I run commands in readme file

# Run a Build

To run a complete build, execute the following commands:

```bash
pelias compose pull
pelias elastic start
pelias elastic wait
pelias elastic create
pelias download all
pelias prepare all
pelias import all -> I got this error here
pelias compose up
pelias test run

Expected behavior Import success pelias map in quebec city

Environment (please complete the following information):

• OS: Ubuntu 22.04 LTS
• Docker versions: Docker version 23.0.0, build e92dd87

Pastebin/Screenshots

[missinglink]

The empty error message is a bug, I'll fix that.

Looking at the code I can see that the error is caused by one of two things:

1. The pelias.json setting imports.whosonfirst.sqlite must be true, this should be the default value but try setting it anyway
2. It can't find a database file matching the regex /whosonfirst-data-[a-z0-9-]+\.db$/ in the path /data/whosonfirst/sqlite, this should have been downloaded in the pelias download all step, you can check in your ./data directory that it exists.

[PhanSon95]

Thank you I imported data to pelias in Quebec success

[PhanSon95]

But now I trying import similar with Canada, I got error when run command

pelias prepare all

This error caused by the .pbf file too large. I got this message and it continue prepare Does it error when I run command pelias import all ?

[missinglink]

What does the error say?

[PhanSon95]

[PhanSon95]

Only show message and continue proccessing. When It done, I doing next step pelias import all

[missinglink]

The polylines error indicates that the openstreetmap PBF file was too large, and therefore it wasn't able to extract data about the street network, as a result you may continue, but your index will be missing all the documents from the street layer (although the address layer will be fine).

If you follow the link in the error message we've provided some information about how to process larger geographies, but I can't provide support for this in the issue tracker.