Support running without root

varesa commented 5 years ago

I am setting up nextcloud instances on an Openshift cluster (Redhat's kubernetes distribution) where the default security context prevents running software as uid=0. I would see that as a good practice in general.

What it would need (at least):

Not binding to privileged ports (e.g. change 80 -> 8080, can be mapped to any host port anyway)

(13)Permission denied: AH00072: make_sock: could not bind to address [::]:80
(13)Permission denied: AH00072: make_sock: could not bind to address 0.0.0.0:80

Making sure apache directories are readable/writable by the non-root user as necessary

[Wed Sep 12 11:06:29.449612 2018] [core:error] [pid 1] (13)Permission denied: AH00099: could not create /var/run/apache2/apache2.pid
[Wed Sep 12 11:06:29.449713 2018] [core:error] [pid 1] AH00100: apache2: could not log pid to file /var/run/apache2/apache2.pid

tilosp commented 5 years ago

I think that this should be fixed in the parent image. Could you open an issue here docker-library/php?

rcdailey commented 5 years ago

I can verify this issue still exists...

  nextcloud:
    image: nextcloud
    restart: always
    user: $UID:$GID
    sysctls:
      - net.ipv4.ip_unprivileged_port_start=0
    environment:
      - APACHE_RUN_USER=#$UID
      - APACHE_RUN_GROUP=#$GID

rcdailey commented 5 years ago

For reference, I opened an issue here:

https://github.com/docker-library/php/issues/743

tilosp commented 5 years ago

should be fixed by https://github.com/docker-library/php/pull/745 and https://github.com/docker-library/php/pull/755

varesa commented 5 years ago

Great! Thanks @rcdailey and @tilosp

Sorry I got busy with life and didn't have time to evaluate and open an issue upstream as you suggested

ghost commented 4 years ago

Is the nextcloud docker image actually capable of running as non root?

rcdailey commented 4 years ago

Is the nextcloud docker image actually capable of running as non root?

Absolutely. I'm running mine that way right now:

app:
  image: nextcloud:16-apache
  restart: unless-stopped
  user: $UID:$GID
  sysctls:
  - net.ipv4.ip_unprivileged_port_start=0
  environment:
  - MYSQL_HOST=db
  - APACHE_RUN_USER=#$UID
  - APACHE_RUN_GROUP=#$GID

(Omitted some irrelevant attributes, such as volume mounts)

ghost commented 4 years ago

Nice! Do you have by any chance pointers to a configuration example for using an nginx reverse proxy outside of the container? And having the nextcloud web server running at a non privileged port (so we avoid the pitfall of allowing <1024 ports being opened by non root).

Is nginx supported inside the container or only Apache with mod_php? I would definitely prefer fpm just for the isolation options.

Thank you!

ghost commented 2 years ago

This is not necessarily 'closed'. @rcdailey I can confirm running the 'app' container works as a non-root user, but the cron container will not operate properly because it relies on using setgroups, which needs CAP_SETGID and is not permitted, so the crond cannot run as a non privileged user as-is. Unless a cron file is installed for the image with the unprivileged user, this opens up a perfectly usable attack vector because cron will execute the Nextcloud maintenance tasks as root, which means the entire framework and associated code (via occ).

This means all you need to do is abusing a bug in Nextcloud and leveraging a second stage that executes through cron. That stage will execute as root. Considering that under the usual Docker model root inside the container is effectively root in the host (and if someone wants to contest this they are not taking in consideration the attack surface accessible in this context), this means running Nextcloud with unprivileged users is an illusion as far as security goes, if using these images/Dockerfiles unmodified.

@tilosp and whoever is relevant. This is a security issue that should be either addressed in the images or properly noted in the documentation/README. People are running these images thinking they provide privilege separation/greater security.

@rcdailey I wanted to also leave a sidenote here: I checked https://github.com/rcdailey/nextcloud-cronjob and that is also not going to address the problem per-se, and it creates one that is even worse. Passing the docker socket to the container opens up a whole host of new issues. The docker daemon socket being made available to the container, is a whole new security gap in its own and makes container to host escalation trivial.

See for example: https://github.com/rcdailey/nextcloud-cronjob/blob/master/scripts/nextcloud-exec.sh https://github.com/rcdailey/nextcloud-cronjob/blob/master/scripts/find-container.sh

Docker administrative access should NEVER be exposed to a container unless this is explicitly required (ex. things like Portainer and similar solutions need docker daemon access, but this is best done through a localhost/loopback channel over the network and that still does not solve the problem of container-to-host escalation).

Do not take this as a personal attack or unjustified criticism, it isn't. These are technical security issues that cannot be shrugged off and need to be addressed.

ghost commented 2 years ago

An easy fix for those who end up here, and want something that is not an extremely unwise security loophole in their setup:

*/5 * * * * docker exec -u UID:GID nextcloud_app_1 php ./cron.php In your host's crontab.

Assuming:

The app container already runs with an unprivileged UID/GID and so does apache (www-data is NOT used). @rcdailey's snippet for the compose file is perfectly sound and valid (https://github.com/nextcloud/docker/issues/458#issuecomment-522273405).
You have removed the cron container from the compose file.

Ideally, edit the crontab for an unprivileged user, but again, if docker is not rootless, any user with docker group privileges can easily escalate to root. So, either use docker rootless, or simply add the line to the crontab for root.

I would suggest using a dedicated LXC or VM guest to run dockerized Nextcloud and its dependencies. Not going down the rabbit hole of "evangelizing" why present day "containers for everything" is foolish, but if you are going to do it, at least leverage some separation.

An alternative: use webcron via another container in the same compose setup, there must be dozens of available unprivileged webhook or periodic web request container images out there.

Hope this helps, and thank you @rcdailey for your work (even despite its faults; security is hard, container security is harder). :-)

rcdailey commented 2 years ago

@vogelfreiheit I implemented my own Docker container for executing cron jobs because of these issues. Please try it and see if it helps.

https://hub.docker.com/r/rcdailey/nextcloud-cronjob

ghost commented 2 years ago

@vogelfreiheit I implemented my own Docker container for executing cron jobs because of these issues. Please try it and see if it helps.

https://hub.docker.com/r/rcdailey/nextcloud-cronjob

@rcdailey Please read both my comments carefully. I'm recommending not using your container purely out of the fact that you are binding the docker daemon socket inside the container. This means whatever executes in your container can execute commands as root in the host (container escape + command execution as root). This is without mentioning additional bugs in the base images or docker itself. Better safe than sorry.

Since you are basically just running something that the host can handle with its own cron daemon, that is what I am suggesting as a safer, sound alternative. You also eliminate the obfuscation/abstraction of the process/cron job.

Like I said, I still appreciate your work, and it is not entirely your fault that Docker as it is offers a false sense of security for many people.

rcdailey commented 2 years ago

I don't have the time to spend on fully reading your comments. If my container didn't work for you, find one that does.

rcdailey commented 2 years ago

Ok I am at my PC now and can sit down and give you the response you deserve @vogelfreiheit. My last comment sounded rude, I apologize for that. I was typing on my phone in the middle of a distracting work day and my brevity didn't come off as polite as it should have.

First thanks for all of your research. I think I need to factory reset the discussion to make sure we're all focused on the right problem here. The whole reason my container exists in the first place is, I argue, because the nextcloud maintainers refuse to support running cron jobs for this inside of the nextcloud container itself. I think this is less of a technical limitation, more of a decision based on principles I disagree with. That is: One container represents one service. At least, when I looked at this years ago, that was one direction the whole conversation took.

I think it would make setting up nextcloud end-to-end a whole hell of a lot easier if they'd just do it. It also addresses all of your security concerns because you can just delete my cronjob service out of your docker compose YAML file. I think if we want to find the best, most constructive way to focus our energy on this issue: It would be getting cronjob support to be a first class citizen in the nextcloud container.

I'm not ashamed to admit my cronjob container is a big fat hack. I took its configurability a few steps above where I originally wanted it to be, in an attempt to make it useful to a wider audience. But please don't misunderstand the objective here. My goal isn't to create this security hardened, robust tool. This is a stopgap solution for an even bigger issue (what I mentioned earlier: lack of official support for this).

I think to address your (very legitimate) security concerns, it would involve adding more complexity, which I'm just quite frankly unwilling to do. My use case for nextcloud is purely homelab. I don't have advanced container orchestration, redundancy, and production issues with hundreds of users. I don't have anyone to worry about exploiting my system since this is all managed by me on a little server sitting in my closet. I can afford certain shortcuts and conveniences because of this.

I don't claim that my container is useful to everyone. I wish it were, but again I need to keep the focus on what is important: I want cronjob support to be in the official nextcloud container so I can decommission my own project. The fact that it exists is a symptom of a much larger issue IMHO.

I hope that helps put things into perspective. This isn't me saying you're wrong (in fact, the opposite: You're 100% right). But I think we're at the point of diminishing returns and we need to focus this energy in a way that is more productive.

EDIT: Small disclaimer. My assertions about the maintainers not supporting cron job intrinsically in the nextcloud container is several years old, based on memory of various conversations in github issues I observed when I researched this. Things could have changed drastically since I last did this research, so I might be completely wrong or misrepresenting them.

ghost commented 2 years ago

Hi @rcdailey, to be honest, we do have to admit they have a point: the external crontab solution I mentioned solves the problem for everyone, it really is that simple (I also don't take credit for it, I'm pretty sure someone else suggested it).

I think you should just add a warning to the container docs, that it exposes the docker daemon socket and the user thus runs it at his own peril. It will also negate anyone running it and blaming you if something goes awry. Also, docker was never meant to be an isolation system. If the docker daemon had some RBAC you could apply, ex. let r's container speak to nextcloud and exec 'specific-command-here', then you would be in the clear. But they don't, ATM. It isn't your fault.

Most homelab users don't need to worry, that I agree with, although ransomware is beginning to hit normal folks.

rcdailey commented 2 years ago

What specific ransomware should homelab users be concerned with? I use this container in isolation, not exposed to any WAN. So I'm not concerned at all. I do agree that a warning is likely necessary, at the very least to make it clear who my intended target audience is and why. Although I can't speak to the security-related aspect you are referring to, as I do not understand much about the practical issues around that.

I take for granted that folks (like you) that value security or would see some issue with how I've done things are already able to identify the risks without me having to explicitly state them. This is a hobby project and I sadly do not have the free time to cater to every possible scenario or situation in which my container may possibly be used (whether that involves some kind of implementation detail or documentation). Any action I'd take involves some cycles I just don't have. Best I can do is add a one-liner in big letters to make it clear this is for homelab only.

dm17 commented 2 years ago

This is not necessarily 'closed'. @rcdailey I can confirm running the 'app' container works as a non-root user, but the cron container will not operate properly because it relies on using setgroups, which needs CAP_SETGID and is not permitted, so the crond cannot run as a non privileged user as-is. Unless a cron file is installed for the image with the unprivileged user, this opens up a perfectly usable attack vector because cron will execute the Nextcloud maintenance tasks as root, which means the entire framework and associated code (via occ).

This means all you need to do is abusing a bug in Nextcloud and leveraging a second stage that executes through cron. That stage will execute as root. Considering that under the usual Docker model root inside the container is effectively root in the host (and if someone wants to contest this they are not taking in consideration the attack surface accessible in this context), this means running Nextcloud with unprivileged users is an illusion as far as security goes, if using these images/Dockerfiles unmodified.

@tilosp and whoever is relevant. This is a security issue that should be either addressed in the images or properly noted in the documentation/README. People are running these images thinking they provide privilege separation/greater security.

@rcdailey I wanted to also leave a sidenote here: I checked https://github.com/rcdailey/nextcloud-cronjob and that is also not going to address the problem per-se, and it creates one that is even worse. Passing the docker socket to the container opens up a whole host of new issues. The docker daemon socket being made available to the container, is a whole new security gap in its own and makes container to host escalation trivial.

See for example: https://github.com/rcdailey/nextcloud-cronjob/blob/master/scripts/nextcloud-exec.sh https://github.com/rcdailey/nextcloud-cronjob/blob/master/scripts/find-container.sh

Docker administrative access should NEVER be exposed to a container unless this is explicitly required (ex. things like Portainer and similar solutions need docker daemon access, but this is best done through a localhost/loopback channel over the network and that still does not solve the problem of container-to-host escalation).

Do not take this as a personal attack or unjustified criticism, it isn't. These are technical security issues that cannot be shrugged off and need to be addressed.

Thanks a lot of analysis. Do you know of any self-hosted alternatives to NextCloud that are already setup to run without root in containers (would have asked via email if I could find yours)? Not trying to derail the topic... Will be trying to run NextCloud completely with de-priv'd docker containers and/or using podman (if that is supported).

ghost commented 2 years ago

@dm17 If you use user: (or --user for non compose deployment) and my suggestion to run cron from the host, that should be fully unprivileged.

You can also experiment with capability drop, ex. remove mknod. However, once the container runs as a fully unprivileged user (create an user with an UID and GID that is far above the range used by regular users, ex. 620x), you should be reasonably good to go without gargantuan efforts.

You might also try your look with unprivileged/rootless dockerd, but it will bring you headaches since most containers unfortunately assume root-privileged dockerd.

tl;dr:

In your host crontab (run crontab -e as the user you have for managign containers, must have docker-group access) */5 * * * * docker exec -u UID:GID nextcloud_app_1 php ./cron.php

In your compose:

app:
  image: nextcloud:FIXME
  restart: unless-stopped
  user: $UID:$GID

Adjust to your setup, container names, etc.

stavros-k commented 1 year ago

For anyone interested, I replaced /cron.sh with

#!/bin/bash

uid="$(id -u)"
gid="$(id -g)"

if [ "$uid" = '0' ]; then
  user='www-data'
  group='www-data'
else
  user="$uid"
  group="$gid"
fi

run_as() {
  if [ "$(id -u)" = 0 ]; then
    su -p "$user" -s /bin/bash -c 'php /var/www/html/cron.php'
  else
    /bin/bash -c 'php /var/www/html/cron.php'
  fi
}

while true;
do
  echo 'Running cron.php...'
  run_as
  echo 'Sleeping for 5 minutes...'
  sleep 5m
done

It works with any user now.

ghost commented 1 year ago

Hi, for anyone dealing with this: you can try the linuxserver image for Nextcloud. Some things like redis and co need to be done manually, but it's quite painless.

These official docker images are extremely unstable for this use case scenario when facing upgrades and I lost interest in the huge mess that major upgrades usually become. Just keep regular backups.

stavros-k commented 1 year ago

Hi, for anyone dealing with this: you can try the linuxserver image for Nextcloud. Some things like redis and co need to be done manually, but it's quite painless.

These official docker images are extremely unstable for this use case scenario when facing upgrades and I lost interest in the huge mess that major upgrades usually become. Just keep regular backups.

How does using an LSIO image solves the issue of "Support running without root"? You DO know that all LSIO based images run as root and drop privileges only when starting processes right? That is a no-no for all kubernetes cluster's that have strict policies. Like runAsNonRoot, readOnlyRootFileSystem.

And also deployments in openshift which afaik does not even give you an option to change that (Could be wrong about openshift stuff)

ghost commented 1 year ago

How does using an LSIO image solves the issue of "Support running without root"?

It doesn't solve the issue for Nextcloud itself, it solves the issue for the users that will lose their data or deal with constant issues when performing major version upgrades because of inconsistent migrations, etc.

I kept a private fork of the docker images for Apache that ran as unprivileged for quite some time, and also for the php-fpm variants (further isolating the PHP processes). It just is not worth it when an alternative exists that does provide sufficient security for users of a product that are arguably not necessarily expert sysadmins. I don't use Nextcloud for anything I care about (I come from a world where people learn how to use Kerberos/NFS safely), but I did help some friends who use it.

You DO know that all LSIO based images run as root and drop privileges only when starting processes right? That is a no-no for all kubernetes cluster's that have strict policies. Like runAsNonRoot, readOnlyRootFileSystem.

I have been doing low-level OS security work with Linux and BSD variants for ~15-20 years, including kernel contributions to different subsystems for both (FreeBSD and Linux)... I think I have a "passing" understanding of the internals of privilege relinquishment in both kernels.

So, to answer your question in technical terms: you seem to engage a fairly forgivable logical fallacy, that is, to think that an initial privilege level is a security risk that cannot be averted... but this is precisely how basically everything in a Linux/*nix system works (including OpenSSH, as a prime example of it). You can safely and irreversibly drop privileges from uid/euid=0 into an unprivileged level that cannot (under any circumstances) revert back to euid=0. The process is described in (https://www.oreilly.com/library/view/secure-programming-cookbook/0596003943/ch01s03.html), however you best reference will be (https://cwe.mitre.org/data/definitions/271.html), more specifically "The Art of Software Security Assessment". Chapter 9, "Dropping Privileges Permanently", Page 479. 1st Edition. Addison Wesley. 2006.

So, the fact that LSIO runs as root early on and relinquishes privileges is entirely irrelevant for the issue at hand: not running gigantic PHP codebases as root or can-root users. While I agree that the ideal thing is a root-less container, odds are most folk using Nextcloud will do so in an environment that does not pose the issues encountered in Kubernetes and co.

I do not make any promises or endorsements about Linuxserver's images, and I certainly don't do it for Nextcloud. It's a fantastic system for millennials and post-millennials that need a web UI for files and other things, and it has an intended audience that benefits from it.... That same audience pragmatically needs something that provides the best compromise in terms of security and stability (not losing their data during/between major upgrades, not messing around occ/database migrations and raw SQL commands to alter tables, etc), so the LSIO stuff is "good enough for them". My personal experience maintaining images that did privilege separation for Apache and the PHP fpm pools has been negative, taking too much time to fix/work around Nextcloud's own lifecycle problems (every major upgrade caused issues) and the fact that they clearly have not developed it with least privilege/security in mind.

Perhaps Nextcloud's developers can put more time into fixing this, but so far it does not seem like a priority of theirs.

TL;DR I can privately provide the source for the changes made to the Apache image, if you want them. But you will learn more useful skills and have a better user experience moving on to a well tested grass roots system like NFS+IDP/Kerberos (surprisingly easy and smooth if you only have Linux users). Don't expect much from the Nextcloud team for this specific issue.

stavros-k commented 1 year ago

I don't disagree that LSIO is good enough for the average Joe. But this issue here was about rootless, official, container, so downstreams don't have to do workarounds to get to their goals.

Another thing is, that while LSIO does drop privileges on the running processes, still the supervisor is running as root inside the container, while also having access to all the mounted data(!).

While kubernetes clusters might not be the cup of tea of the average Joe, Nextcloud is a software that mostly targets small and/or bigger businesses and mainly at scale! So, yea Kubernetes security policies and deployments tactics have to be considered. LSIO is not containers built to scale, but built as an "easy" to deploy container. (Nextcloud now have an AIO 'stack' that people can take advantage. If ease of deployment is the goal.)

Again, I'm not hating on LSIO, it's has it's audience. But it's not for everyone.

Also either you deploy Official or LSIO or whatever, you will have to mess with occ or manually editing files to tune/configure nextcloud. Not everything is in the UI.

TL;DR the goal here is not to tell people "use X software instead of Y" but to solve the issue at hand. Sure there are ton of alternatives way to get to the town, but I wanna take the bus!

Ornias1993 commented 1 year ago

So, to answer your question in technical terms: you seem to engage a fairly forgivable logical fallacy, that is, to think that an initial privilege level is a security risk that cannot be averted

The point was not that LSIO is insecure (it's only slightly less secure), the fact is that many professional deployment options don't want containers requesting root permissions, even initially. This is also intended to prevent some levels of supply chain attacks, as pods can be enforced to not run as root in a multitude of levels.

Also: Painting simple misunderstandings between parties as logical fallacies is kinda weird.

ghost commented 1 year ago

The point was not that LSIO is insecure (it's only slightly less secure), the fact is that many professional deployment options don't want containers requesting root permissions, even initially. This is also intended to prevent some levels of supply chain attacks, as pods can be enforced to not run as root in a multitude of levels.

Sounds like those same people would (or should) not pick Nextcloud as a container based solution for a multitude of reasons.... let alone run it on Kubernetes or a similar system.... If someone wants to use an astronaut suit for diving surely they can try, it still does not seem like a good idea.

We are entering subjective territory, though, but my point still stands: an organization with stringent security needs will not and should not use Nextcloud's container images... per se, but especially not run them in a container cluster. Even in terms of performance and storage access latency it isn't a great idea. Storage servers don't benefit from degrees of separation from the IO layer (hence why nfs or kernel-mode CIFS are the go-to solution in the enterprise world.... yes, even NFS-driven home directories).

You are all welcome to get this issue moving forward, though. If you can get the Nextcloud developers to care/fix it after all this time since 2018 I will do some cartwheels in celebration.

Painting simple misunderstandings between parties as logical fallacies is kinda weird.

I think the sarcasm/humor wasn't flying that high over anyone's head.

Ornias1993 commented 1 year ago

The point was not that LSIO is insecure (it's only slightly less secure), the fact is that many professional deployment options don't want containers requesting root permissions, even initially. This is also intended to prevent some levels of supply chain attacks, as pods can be enforced to not run as root in a multitude of levels.

Sounds like those same people would (or should) not pick Nextcloud as a container based solution for a multitude of reasons.... let alone run it on Kubernetes or a similar system.... If someone wants to use an astronaut suit for diving surely they can try, it still does not seem like a good idea.

We are entering subjective territory, though, but my point still stands: an organization with stringent security needs will not and should not use Nextcloud's container images... per se, but especially not run them in a container cluster. Even in terms of performance and storage access latency it isn't a great idea. Storage servers don't benefit from degrees of separation from the IO layer (hence why nfs or kernel-mode CIFS are the go-to solution in the enterprise world.... yes, even NFS-driven home directories).

You are all welcome to get this issue moving forward, though. If you can get the Nextcloud developers to care/fix it after all this time since 2018 I will do some cartwheels in celebration.

Painting simple misunderstandings between parties as logical fallacies is kinda weird.

I think the sarcasm/humor wasn't flying that high over anyone's head.

Your argument failed the second you started to act like all industry standard container deployment tools where “super niche” for some reason.

ghost commented 1 year ago

Your argument failed the second you started to act like all industry standard container deployment tools where “super niche” for some reason.

Not quite. Your "industry standard container deployment tools" were not developed for you to "deploy" Nextcloud, neither was Nextcloud designed to scale or perform well in such environments, for that matter. It is a niche use.

Judging by your account and repository history you seem to take personal offense at someone calling your "toys", well, toys. Instead of derailing this issue into some sort of protoautistic pissfight (by itself no good use of our time), let's stick to using it for its intended purpose.

Feel free to submit a PR to the Nextcloud folks with your solution.

Ornias1993 commented 1 year ago

Yeez this is like the 3th personal attack of yours in a row?!

if you call tools like openshift “toys”, I really dont think you should be mixing into this discussion at all.

for transparity: i dont personally use an enterprise-grade hardened deployment system.

however, calling kubernetes deployments niche is simply absurd. But saying nextcloud docker isnt build for it at all, is simply compleetly wrong as well. Multiple fixes for things like updates on scaled deployment exist fir a reason.

even so: no one here talked about scaling at all, thats your strawman…

nextcloud / docker

Support running without root #458