Security considerations

[archfz]

This projects looks good and all. But the docker version really needs documentation about it's security. The mount of the docker socket can potential open up huge attack vectors. We need to be ensured that code running in the container is safe and all around secured.

So the question is what can this project provide in terms of security of the underlying code? Does the code check for potential malicious commands executing on the docker socket? What about other processes running in the container? And recommendations to secure the container that need to be done by the user?

An idea might be disallowing external network access. https://docs.docker.com/engine/reference/commandline/network_create/#network-internal-mode

https://docs.datadoghq.com/security_platform/default_rules/cis-docker-1.2.0-5.31/

Maybe it should also be considered if the docker version of this package should be abandoned.

[maietta]

By design, the project can run jobs inside or outside the container with the appropriate syntax. Like cron, whoever has the ability to edit a crontab schedule is likely going to have some say in what runs and how it's run in that environment. The same holds true for Ofelia (or my fork called Chadburn).

Whoever has the ability to set Service labels on Docker services, probably has some say in the deployment of services.

Like anything else, there are probably multiple attack vectors that can be looked at for any given piece of software. One attack vector can possibly be to craft a malicious payload in the form of a string in the key and value pairs for service labels. That would be a Docker bug and not a bug in Ofelia.

I'm curious what creative ways people might have to exploit Ofelia from a service label. I cannot see a sane scenario where an end-user, such as a web hosting client would have the ability to define the syntax of what can be executed. In that scenario, I would hope that some sanity checks are in place to make sure jobs can only be run in the containers they own or to be able to run services within their environment, where the userspace is tightly jailed.

[archfz]

I really like the architecture where I can just ship cron jobs as code, without need of host configuration if docker is already available. But the security issues that come with it really scratch my brain.

My only concern about labels is that these can be added on image build as well. Thus using public base images could be a threat. One could fabricate a command to gain remote terminal access inside a running job. So is there any way to prevent this, can build labels be differentiated from runtime labels ?

My main and most important concern is that ofelia container is running with root user. If somehow someone could gain access to executing commands inside of the container, for say remote execution, that would mean total control of the host due to the access to the docker socket. Disallowing all network traffic to and out to the container could reduce the issue considerably. But best would be if the container was not running as root. Question is if this is possible and how, as ofelia needs to access the socket.

Maybe an idea would be to have the main process ofelia run as a different user, while there would be an SUID wrapper over the communication with the docker socket. This wrapper could then be protected by root only access and the main process would not be able to alter it. The wrapper could also have embedded protection to disallow access of executing any docker command, would allow only running in containers the commands that are specified by the labels.

(PS: I feel like I should also take down this comment due to security. Please advise.)

[maietta]

Hi, @archfz. An open conversation about security can benefit all. Security through obscurity is hardly security at all. Let's keep the conversation available for others to see. Perhaps we'll get a security nut sharper than me to help us solve this one.

I've used program wrappers in the past to do exactly what you are talking about and that could mitigate some of your concerns.

I maintain a fork of Ofelia called called Chadburn, which I started working on to enable it to work out of the box as a One-Click-App for Caprover, a DIY PaaS hosting solution for Docker Swarm environments. Your comment got me thinking in terms of how a wrapper could be implemented in a scenario like that. Ideally, a solution should be crafted that takes into consideration how it's installed.

EDIT: Removed link to article as it actually discusses letting non-root users access root privilege's, not the way I was thinking at first. We need ability to prevent direct root calls (like using a traditional wrapper). I am looking for examples of implementations in Go.

[maietta]

It is possible to run Docker in "rootless" mode as discussed here: https://docs.docker.com/engine/security/rootless/. This may be a solution for a few out there, but it's definitely going to not be a solution for the rest of everyone.

[maietta]

Hi @archfz. Hope you're doing well. I've been thinking about this problem further and believe this problem is beyond the scope of Ofelia or Chadburn's design scope. The idea is it listens to service changes to container starts/stops and looks for service labels to take action.

With that in mind, as I write the documents for Chadburn, I will be introducing security concerns and possible solutions that go beyond the function of the scheduler. This will help keep the program maintainable while teaching people about how to better deal their systems design.

In am in the process of rewriting Chadburn (fork of Ofelia) from scratch and pouring over issues like yours as part of my development process. I thank you for bringing this issue to light. Issues like yours will help improve the documentation.