pelias / docker

Run the Pelias geocoder in docker containers, including example projects.
MIT License
338 stars 226 forks source link

How to import multiple countries ? #128

Open bidabil opened 5 years ago

bidabil commented 5 years ago

Hey team!

As it was written on the title. Are there ways to import more countries?

If yes, should I write a new project called for example "MultipleCountries" and modify it's pelias.json. The modification is about adding more PBF files file so that it can be downloaded and imported.

is it the same things for "geonames","openaddresses" ... etc ? if no, what kind of solution should I take ?

an other point ... I'm trying to importe Algeria, Tunisia, And Morroco only ... is an 2 vCPU with 4Giga of Ram and 250SSD adequated ?

Thanks a lot.

orangejulius commented 5 years ago

Hi @bidabil, Thanks for writing in. Importing multiple countries is certainly possible.

It would be great if we had a go-to example, so if you make an Algeria, Tunisia and Morroco project, that's as good as any. You can name it after the countries, and we'll highlight it as a multiple country example in the readme.

Here's how it would work:

For OSM, both the download and import sections are arrays that can have as many entries as you'd like, as shown in the readme.

For Who's on First, the importPlace parameter can be an array of as many WOF IDs as you'd like. One ID for each country would be best, and you can find them in the WOF Spelunker.

For OA, it's a bit annoying but you have to make a list of all the files from each country. There is a full list on the OA website.

The Geonames importer does not currently support multiple countries (there is a partially completed pull request), but Geonames data is mostly imported into WOF now, and we are actually considering dropping support for it these days, as it has all been imported into WOF.

The Polylines importer is also used to bring street data in from OSM. It only accepts a single file right now, but the files are a relatively simple format. You could generate a polyline file for each of the 3 countries an concatenate them together with cat (cat country1.0sv country2.0sv country3.0sv > threeCountries.0sv, for example).

Let me know if you have any trouble getting all that set up.

P.S. 2 vCPU and 250GB disk should be fine, but 4GB RAM will probably not be enough. Go for 8GB to be safe if you can.

lucjans26 commented 3 years ago

Hey @orangejulius,

I recently needed to do an import of, basically, western Europe (Netherlands, Belgium, Luxembourg, Germany, France, Spain, Portugal). Based on the information in this thread I managed to create a working configuration. I left out the Geonames config for all but the original config that was already there. I also did not create the custom Polyline. Aside from these two points, the geocoding seems to be functioning as expected in my use-case. I noticed that the projects folder still does not contain a multi-country import. Considering this, I want to share my config as an example for anybody that wants to do a multi-country import themselves.

peliasMultiCountryConfig.zip

ShahanAli2239 commented 10 months ago

Hey @lucjans26

I recently working for Algeria and Tunisia and i have to configure Algeria and Tunisia datasets instead of All, what should i do? please assist