Improved default security for Pelias service dependencies (Elastic Search, etc)

[Jezternz]

Use-cases

Currently if you deploy Pelias Docker containers to a public facing space, there is a good chance you will stumble into a world of pain. In my specific case I setup Pelias, a week later had a ransomware attack, removing all indexes (and presumably bitlockering them). Although this does feel like an administrative problem, there are simple changes to the docker-compose files that could resolve this.

Proposal

I believe it would make sense to have a reasonable (docker) network access policy by default that only exposes the pelias api itself. This would require changes to the docker-compose file, what I had envisioned is basically described here.

I would expect the pelias api to be publicly exposed (as it is effectively read-only) and all the other (vulnerable) services (such as elasticsearch) to be exposed only on a private network. I realize this will limit access to the non-pelias API's, and may add a barrier for debugging, however this feels like a reasonable price to pay given how vulnerable the docker containers are in their default state.

Maybe a reasonable solution would be to use an optional env parameter named something like "SERVICE_DEPENDENCIES_PRIVATE=" with a default of true (or false if that's preferable), this could sit beside DATA_DIR= in the .env file. Additionally this would require a change to the dockercompose.yml to be something like:

networks:
  net-pelias-deps:
    driver: bridge
    internal: '${SERVICE_DEPENDENCIES_PUBLIC}'
  net-pelias-api:
    driver: bridge
api:
    ...other config
    networks:
      - net-pelias-deps
      - net-pelias-api
...other services
    ...other config
    networks:
      - net-pelias-deps

References

Other example of someone who suffered a similar fate

[missinglink]

Hi @Jezternz have a look at https://github.com/pelias/docker/issues/227#issuecomment-738859618

I would be :+1: with adding some notes to the documentation about steps that could be taken to strengthen security but we would like the defaults to be set up for ease of development.

[missinglink]

Is there some reason why you don't have a firewall with port blocking rules protecting your production environment?

[Jezternz]

Thanks for the replies @missinglink, I am very grateful for Pelias and all the hard work the contributors have done, it is an amazing OS project!

Hi @Jezternz have a look at #227 (comment)

I would be 👍 with adding some notes to the documentation about steps that could be taken to strengthen security but we would like the defaults to be set up for ease of development.

I was thinking that it could be behind the env var, and default it to SERVICE_DEPENDENCIES_PRIVATE=false if preferable, but I understand that this would require additional maintenance, testing etc so may not be preferable. I will take a look at see if I can find a clean way to add the security post-pelias' install, and post here if I have any luck.

Is there some reason why you don't have a firewall with port blocking rules protecting your production environment?

At this stage I have only been playing around with a system that is not yet in production, though am reasonably new to he devops space (especally unix), and expected a machine-level configuration would be sufficient (this server is a standalone machine that runs pelias, isolated from the primary application - and the API itself sits behind a reverse proxy that requires auth). My understanding was that unix iptables was a typical way to block/allow traffic, and docker opened it all up by default. I guess you are referring to a firewall external to the machine itself? (in which case I have not considered it, given the small amount of infrastructure I am running in that space - but maybe I need to)

Thanks

[orangejulius]

Hi @Jezternz, Thanks for bringing this up.

We definitely want to balance security and ease of debugging. In a previous discussion one option considered was setting the Docker port configuration so that some (or all) containers were accessible only on localhost, rather than all interfaces. Especially for Elasticsearch that seems like a reasonable compromise.

We'd welcome a PR to do that, as I think it's pretty low impact.

That's a lot simpler than going down a path with any sort of environment variables.

We should also add something to the documentation mentioning the intent of this project (local development), and at least touching on the potential security concerns.

[Jezternz]

Thanks @orangejulius for the reply, this seems like a reasonable way forward, I will look into a pull request.

We should also add something to the documentation mentioning the intent of this project (local development), and at least touching on the potential security concerns.

Regarding the docker repo being meant for local development, forgive my naivety but I was wondering why this is the case, the docker containers provided seem like an incredibly convenient way to wrap up a production system. Is it more that the goal of this repo is for development and not explicitly to satisfy production requirements? so not so much that it could not be used in production, but just that it has not been especially considered?

[Jezternz]

Well I have forked the repo, made the very basic changes as you described @orangejulius on #252, and they seem to be working as expected. I can access elasticsearch via curl on the machine, but not externally (done by prepending the localhost ip to the port config in the docker-compose files). Building Pelias Australia Project overnight, so provided things look good, I will create a pull request tomorrow :)

Thanks to both of you for all the work you have done around pelias (I looked up the Geocode Earth team), I was pretty dissapointed with the availability before finding pelias. @missinglink you don't happen to be a fellow kiwi? :D

[missinglink]

Hah guilty @Jezternz I usually live in Berlin but I've absconded with my family to Wellington until the northern hemisphere returns to some form of normality ;)