search

pelias / pelias

Pelias is a modular open-source geocoder using Elasticsearch.
https://pelias.io
MIT License
3.15k stars 223 forks source link

Alternative of WhosOnFirst #815

Open toton6868 opened 4 years ago

toton6868 commented 4 years ago

I have a better Dataset for my country which 5 hierarchical administrative Layer. But in WhosOnFirst is based on the generalised dataset. My Dataset in Shapefile format. Can I use this shapefile in PIP service instead of WhosOnFirst?

I was trying to integrate Pelias-shp-importer with no luck.

Thanks

NickStallman commented 4 years ago

I've got similar issues with Australia where the WOF data is out of date.

I'm working with them to update WOF with the newer authoriative data which is easier than trying to make Pelias use a different data source.

toton6868 commented 4 years ago

Would you please kindly elaborate how you are doing this? It will be a great help for me

NickStallman commented 4 years ago

Admittedly it's not the easiest thing to do, however WOF is just a whole bunch of GeoJSON files so making an edit isn't difficult.

I've had to make my own scripts to take input data as GeoJSON (or convert SHP to GeoJSON), then match it against the WOF data. Then I make my edits to the WOF data and run it through the py-mapzen-whosonfirst-export/wof-exportify tool to tidy the result up in to the WOF style format.

Unfortunately my scripts aren't really clean enough for sharing, and also rely on a MongoDB database to assist with the matching steps (MongoDB can do spatial searches against GeoJSON natively).

toton6868 commented 4 years ago

I can able to convert the My Shapefile to any format of the world, so converting GeoJSON will not be a hectic task for me. So where I am facing a problem? Matching my data against WOF and replace existing WOF data. I am using WOF SQLite format. There is 5 tables with a lot of columns. The problem is here. I can convert shapefile to geojson but how can make it like that SQLite?

NickStallman commented 4 years ago

@toton6868 the SQLite databases are just for using the raw data with tools like Pelias. They are actually built from a git repository.

You can find your country's respository here: https://github.com/whosonfirst-data/ You'll see that everything is plain GeoJSON files there and the SQLite databases are generated periodically from those repositories.

toton6868 commented 4 years ago

Yes I know. I am getting my country data from there but as like you the data of my country is not good enough to use and I have purchased far better data

NickStallman commented 4 years ago

If you've purchased non-free data then that makes it slightly more complated since you won't be able to contribute it back to WOF which is a pity.

If you just want to use your data and not anything else from WOF you'd want to make your own WOF-style folder of GeoJSON files from scratch. Pelias can use both GeoJSON and SQLite formats so I'd probably stick to the GeoJSON and then you just need the format the attributes to match the WOF format.

missinglink commented 4 years ago

We're actively working to support multiple data sources for boundary data.

Some of the domain modelling work has been completed in https://github.com/pelias/spatial but that repo is still in early stages of development.

Have a play with the demo, you'll be able to see a few different data providers in the GUI. Unfortunately, we don't have a timeline for delivery of that feature, but I'd expect it wouldn't be during 2019.

For now, the best solution is to convert the data you have into a format which resembles WOF and then use it as if it were.

If you go that route then be aware that the boundary data is used in multiple places within the system, as it is imported to elasticsearch as documents for search, it's also imported into 'placeholder' and the 'pip' service.

toton6868 commented 4 years ago

"For now, the best solution is to convert the data you have into a format which resembles WOF and then use it as if it were."

@missinglink I was trying to do the same thing. But probably missing something silly. Can you give me hint or provide me a link where I can find the exact format of WOF JSON file? I can convert my data into any given format. In WOF repo there is a folder based format or SQLite format. I am confused a little bit what I am missing or where to start. Thanks