Docker: Run Image as non-root

[christopherscholz]

Is your feature request related to a problem? Please describe.

Currently the Image is not set up for running it as non-root user. Many things have to be changed for it to be possible.

It is best practice to run any container as non-root in production. This is to

• increase security
• avoid platform restrictions -> e.g. openshift does not allow containers running as root user

There is a good article about it by bitnami.

But there are some use cases, which limit the possible options

• As a data engineer I need to be able to mount a host dir (e.g. mage project code) and access it with rwx privileges in order to run the docker image
• As a data engineer I need to be able to mount my previously created volume with owner root and access it with rwx privileges
• As a data engineer I don't want to build the image just to be able to run it as non-root.

Describe the solution you'd like

General

• Dont setup a specific username. But instead use any nameless and homeless (user without a home directory or name, just a UID) user
  • This allows us to map the host uid to the container uid, which means that it doesn't need root privileges to write to the hostpath
• Setup everything as root user
• Make sure that python tmp directories are set in a writable path for non-root (e.g. setting XDG_CONFIG_HOME)
• After setup is completed change to e.g. USER 1001
  • This choice can be changed by docker run, docker compose, kube securitycontext, .. to any uid
  • Remaining privileges can be dropped e.g. using docker run --cap-drop=all
• Make sure there is only a couple of writetable locations, in order to set the container as readonly and mount volumes to writeable locations.

Python

• set up as normal
• add virtual env, which inherits system packages
• use virtual env as default python
• add correct read, write, execute permissions for the Others(chmod) class for this virtual env This is important to allow users adding new packages, but not change mage base packages

R

• set up as normal
• add additional library/package path for user added packages
• add correct read, write, execute permissions for the Others(chmod) class for the additional paths This is important to allow users adding additional packages, but not change mage base packages (pacman, renv)

Using an anonymous user without home or name allows us to overwrite in kubernetes security context, docker run or docker compose definition. Developers can still use root user for their local development.

Describe alternatives you've considered

Setting up the non-root user as a user with home and name

• Will have issues when mounting a host volume, as the host uid and container uid will not match

Setting uid as argument for the build and set up the non-root user as a user with home and name

• Would need to rebuild the image for each deployment, as the user is set up during build time and the only way to change the user would be by providing a different uid to the argument and rebuild it. Inexperienced docker users, would have problems

Additional context

This must not break previous deployments. This is part of the mage roadmap. This is replacement for unfinished #2064 Slack conversation regarding implementation details. cc: @wangxiaoyou1993

[dy46]

hey @christopherscholz are you working on this issue?

[gaspardc-met]

Hey @christopherscholz ,

Thank you for the Docker epic upgrades, much appreciated. Do you plan on moving on with the non-root improvement yourself ?

The issue was raised on my side, preventing me from using mage anymore for production @dy46 : any way the team could handle this ?

If all else fails, I could maybe look into it but I don't know much about permissions

[wangxiaoyou1993]

Added the doc for building custom docker image to run Mage as non-root users: https://docs.mage.ai/production/custom-docker-image/non-root-user