search

pelias / pelias

Pelias is a modular open-source geocoder using Elasticsearch.
https://pelias.io
MIT License
3.21k stars 222 forks source link

Simplify installation #61

Closed missinglink closed 7 years ago

missinglink commented 9 years ago

One of the core goals of the project is to provide an easy installation process.

Due to several factors the install instructions have grown to cover several pages: https://github.com/pelias/openstreetmap

I propose that we provide a pelias binary file which handles configuring, testing settings are correct and running of imports.

If there are any other ways of reducing complexity we should explore them also.

sevko commented 9 years ago

Isn't this covered by pelias/vagrant? @heffergm 's done a great job there, and it provides the easiest way for people to get started with a local Pelias environment while also documenting how to manually set it up in a production ops setting (for power users, that is).

sevko commented 9 years ago

Looks like we should update INSTALL.MD to mention Vagrant.

heffergm commented 9 years ago

Yes, the vagrant environment provides a means of locally setting something up (ignoring the fact that it's now rendered quite a bit less useful given the necessity of loading all quattro into memory, which makes the footprint too large to be run by most people locally I suspect).

It does not, however, simplify the installation process itself, which is rather convoluted. I think it's fair to say that reading through all the chef code to try and divine what's happening is a less than pleasurable experience for a user who is used to one-and-done binaries in the osm world (admittedly an apples-to-oranges comparison, but I think it's relevant if we want people to be able to deal with the fact there are a ton of repositories to clone, configs to set up, data to load... all in a not very uniform manner at the moment).

The idea that a single binary could provide a means of interfacing with all the disparate packaging is a good one in my mind.

sevko commented 9 years ago

Fair enough. I'm all for a standalone binary for building the index, starting the API, even running imports, etc.

sevko commented 9 years ago

Right, doing something like:

$ pelias schema create_index
$ pelias openaddresses --deduplicate --admin-values
$ pelias geonames -i US
$ pelias openstreetmap#experimental
$ pelias api serve
$ pelias schema drop_index

would be pretty amazing.

sevko commented 9 years ago

This should be (partly) resolved by pelias/pelias-cli.

sevko commented 9 years ago

Is there any more to this issue, besides tutorials and the like?

dianashk commented 9 years ago

As far as I understand, the CLI doesn't address the ES installation or checking for other expected dependencies. It would be nice to add another command like pelias setup.

missinglink commented 9 years ago

There are still a lot of steps in https://github.com/pelias/openstreetmap that could be made easier.

sevko commented 9 years ago

Indeed. Addressing @diana's request should go some way to making them simpler.

dianashk commented 9 years ago

Let's revisit this in the near future. It's a relatively involved story (adding appropriate label) and should be thrown on-deck when there is sufficient time to focus on it.

flotpk commented 9 years ago

Hi All, I am searching for the setup guide for a production environment and reading through the conversation above, @sevko mentioned that pelias/vagrant (by @heffergm) has documentation on "how to manually set it up in a production ops setting (for power users, that is)" - but searching through https://github.com/pelias/vagrant, couldn't find anything on manual setup - need someone to point me to the documentation if it exists.

Also would like to find out if anyone have try out vagrant + docker for setting up Pelias ?

sevko commented 9 years ago

I misspoke a bit. Basically, the Chef recipes in that Vagrant setup are the canonical "documentation" of how to set things up step-by-step, complete with external dependencies, version numbers, and all that good stuff. We'll be getting some human documentation rolled out really soon.

re: docker, @heffergm ?

missinglink commented 9 years ago

@flotpk we are still in beta and so would advise using the vagrant images and rebuilding regularly at this point, we don't have a 'production' build guide and probably wont have for some time. Feel free to use the planet-wide hosted service at http://pelias.mapzen.com until that is available, the rate limits are generous.

If you would like to experiment and install it yourself from scratch in order to better understand how it works and to modify it to suit your specific needs, I would recommend following this guide https://github.com/pelias/openstreetmap, I wrote that for the openstreetmap importer but once you have OSM set up and importing the rest is easy to build.

I managed to get Pelias running on a raspberry pi v2 using that install over the weekend so I can confirm it's current.

heffergm commented 9 years ago

From an operational perspective, Pelias is a workflow with a main backend dependency of elasticsearch.

If you want to build an Elasticsearch cluster capable of running a planet import of Pelias using Docker, Elasticsearch would be the place to go for that. I know they provide a very basic docker recipe for running a single node.

Setting up the rest of the dependencies and workflow is going to be a matter of determining what works best for you in your particular environment. As mentioned above, the chef recipe documentation for vagrant should give you an idea of what a workflow might look like. Personally, I don’t think replicating that in docker is the way to go.

Grant Heffernan

Il giorno 11/mag/2015, alle ore 09:51 AM, Severyn Kozak notifications@github.com ha scritto:

I misspoke a bit. Basically, the Chef recipes in that Vagrant setup are the canonical "documentation" of how to set things up step-by-step, complete with external dependencies, version numbers, and all that good stuff. We'll be getting some human documentation rolled out really soon.

re: docker, @heffergm https://github.com/heffergm ?

— Reply to this email directly or view it on GitHub https://github.com/pelias/pelias/issues/61#issuecomment-100915618.

flotpk commented 9 years ago

Thanks @sevko, @missinglink and @heffergm for your comments, advice and links. Really helpful to me. Cheers, Florence

flotpk commented 9 years ago

Quick question before I go down the path of manual installation - is it possible to run the current "recipes" in an empty VM (e.g. ubuntu trusty) instead of using vagrant + chefdk to spin up a new VM and perform the auto installation ? If yes, how much changes is required in the "recipes" and do I use chef-solo to run the "recipes" ??

heffergm commented 9 years ago

There's nothing vagrant specific about any of the chef code.

On Tue, May 12, 2015, 5:07 AM flotpk notifications@github.com wrote:

Quick question before I go down the path of manual installation - is it possible to run the current "recipes" in an empty VM (e.g. ubuntu trusty) instead of using vagrant + chefdk to spin up a new VM and perform the auto installation ? If yes, how much changes is required in the "recipes" and do I use chef-solo to run the "recipes" ??

— Reply to this email directly or view it on GitHub https://github.com/pelias/pelias/issues/61#issuecomment-101204253.

irony commented 9 years ago

An example docker-compose.yml file would be a great way of documenting the setup.

juanpujol commented 8 years ago

Is there anything new on this? Docker maybe? The vagrant docs said that it probably won't work because of ElasticSearch 2.

Thank you.

dianashk commented 8 years ago

Unfortunately we haven't had the bandwidth to tackle this yet. We'd really love to checkout what others have setup locally to solve this problem.

dmitry commented 8 years ago

Does anyone would like to join to complete this task?

hannesj commented 8 years ago

You can take a look at our Docker-based setup. The Dockerfiles are included in:

https://github.com/HSLdevcom/pelias-api https://github.com/HSLdevcom/pelias-data-container

However, they are quite specialized in our implementation for geocoding within Finland.

easherma commented 8 years ago

What does the team see as the future direction installation is moving in? Seems like the vagrant repo has been difficult to keep updated. So we see moving to Docker? I've been doing manual set-ups, which isn't too bad. Using the Chef scripts outside of the vagrant image is an interesting idea, and not one I considered previously (I assumed they were tied more directly to the vagrant image itself).

orangejulius commented 8 years ago

I'm happy to report there has been increasing activity around Docker on the team. We've all dug into it a bit and the new https://github.com/pelias/interpolation repo already has a Dockerfile!

It will be a long time before everything uses Docker, but I think that's the direction we're heading in right now.

dianashk commented 7 years ago

https://github.com/pelias/dockerfiles is now ready for use and we are working to complete kubernetes scripts that will use the same Dockerfiles as the docker-compose setup. Simple installation accomplished! 💥

dmitry commented 7 years ago

@dianashk thanks for working on this!

heidenthal commented 5 years ago

I followed the Docker instructions and it didn't work at all. I would really like to use Pelias, but without a clear (and updated) way to install, it will never happen.

I spent over 6 hours over two different attempts to get the installation working. I had to diverge from the instructions in several places because steps were missing or changed. Unfortunately, not being familiar enough with the project, I couldn't get to a functional state.

orangejulius commented 5 years ago

Hi @heidenthal, This issue is pretty old, so hopefully you stumbled upon our current docker instructions at http://github.com/pelias/docker/.

We generally try to keep it up to date but if you ran into any problems, we would appreciate if you could open an issue in that repository detailing what went wrong. We should be able to help you get on track and ensure the documentation is clear going forward.