glotcode / docker-run

Api for running untrusted code in containers
MIT License
108 stars 25 forks source link

docker-run

Overview

docker-run provides a http api for running untrusted code inside transient docker containers. For every run request a new container is started and deleted. The payload is passed to the container by attaching to it and writing it to stdin. The result is read from stdout. The communication with the docker daemon happens via it's api over the unix socket. This is used to run code on glot.io.

Api

Action Method Route Requires token
Get service info GET / No
Get docker info GET /version Yes
Run code POST /run Yes

Docker images

When a run request is posted to docker-run it will create a new temporary container. The container is required to listen for a json payload on stdin and must write the run result to stdout as a json object containing the properties: stdout, stderr and error. The docker images used by glot.io can be found here.

Performance

The following numbers were obtained using glot-images on a 5$ linode vm running 'Hello World' with httpstat multiple times locally on the same host and reading the numbers manually. Not scientific numbers, but it will give an indication of the overhead involved.

Language Min Max
Python 250 ms 350 ms
C 330 ms 430 ms
Haskell 500 ms 700 ms
Java 2000 ms 2200 ms

With gVisor (optional)

Language Min Max
Python 450 ms 570 ms
C 1300 ms 1500 ms
Haskell 1760 ms 2060 ms
Java 4570 ms 4800 ms

Security

Docker containers are not as secure as a vm and there has been weaknesses in the past where people have been able to escape a container in specific scenarios. The recommended setup is to store any database / user data / secrets on a separate machine then the one that runs docker + docker-run, so that if anyone is able to escape the container it will limit what they get access to. That said, glot.io has been running untrusted code in docker containers since 2015 without any issues.

Depending on your use-case you should also consider to:

Installation instructions

FAQ

Q: How is fork bombs handled?

A: The number of processes a container can create can be set with the DOCKER_CONTAINER_ULIMIT_NPROC_HARD variable.

Q: How is infinite loops handled?

A: The container will be killed when the RUN_MAX_EXECUTION_TIME value is reached.

Q: How is large output handled?

A: Docker-run will stop reading the output from the container when it has read the number of bytes defined in RUN_MAX_OUTPUT_SIZE.

Q: How is high memory usage handled?

A: The max memory for a container can be set with the DOCKER_CONTAINER_MEMORY variable.

Environment variables

Required

Variable name Type Description
SERVER_LISTEN_ADDR <ipv4 address> Listen ip
SERVER_LISTEN_PORT 1-65535 Listen port
SERVER_WORKER_THREADS <integer> How many simultaneous requests that should be processed
API_ACCESS_TOKEN <string> Access token is required in the request to run code
DOCKER_UNIX_SOCKET_PATH <filepath> Path to docker unix socket
DOCKER_UNIX_SOCKET_READ_TIMEOUT <seconds> Read timeout
DOCKER_UNIX_SOCKET_WRITE_TIMEOUT <seconds> Write timeout
DOCKER_CONTAINER_HOSTNAME <string> Hostname inside container
DOCKER_CONTAINER_USER <string> User that will execute the command inside the container
DOCKER_CONTAINER_MEMORY <bytes> Max memory the container is allowed to use
DOCKER_CONTAINER_NETWORK_DISABLED <bool> Enable or disable network access from the container
DOCKER_CONTAINER_ULIMIT_NOFILE_SOFT <integer> Soft limit for the number of files that can be opened by the container
DOCKER_CONTAINER_ULIMIT_NOFILE_HARD <integer> Hard limit for the number of files that can be opened by the container
DOCKER_CONTAINER_ULIMIT_NPROC_SOFT <integer> Soft limit for the number of processes that can be started by the container
DOCKER_CONTAINER_ULIMIT_NPROC_HARD <integer> Hard limit for the number of processes that can be started by the container
DOCKER_CONTAINER_CAP_DROP <space separated list> List of capabilies to drop
RUN_MAX_EXECUTION_TIME <seconds> Maximum number of seconds a container is allowed to run
RUN_MAX_OUTPUT_SIZE <bytes> Maximum number of bytes allowed from the output of a run

Optional

Variable name Type Description
DOCKER_CONTAINER_READONLY_ROOTFS <bool> Mount root as read-only (recommended)
DOCKER_CONTAINER_TMP_DIR_PATH <filepath> Will add a writeable tmpfs mount at the given path
DOCKER_CONTAINER_TMP_DIR_OPTIONS <string> Mount options for the tmp dir (default: rw,noexec,nosuid,size=65536k)
DOCKER_CONTAINER_WORK_DIR_PATH <filepath> Will add a writeable tmpfs mount at the given path
DOCKER_CONTAINER_WORK_DIR_OPTIONS <string> Mount options for the work dir (default: rw,exec,nosuid,size=131072k)
DEBUG_KEEP_CONTAINER <bool> Don't remove the container after run is completed (for debugging)