Kubernetes cannot run mariadb docker container as an arbitrary user

[zelong430]

Hi,

I'm trying to deploy a mariadb instance in our enterprise on-perm Kubernetes cluster. The challenge is that our Kubernetes cluster only allow me to deploy pods with a fixed default security context (a fixed specific UID/GID). Therefore, I can't run the pod as root or as any users other than the specific UID/GID allowed. The docker file specific set /var/lib/mysql permission to user mysql:mysql. So when I launch the pod through default docker entry point, mysqld always complains not allowed to created files in /var/lib/mysql due to permission.

I found this issue https://github.com/docker-library/mariadb/issues/304. Seems like there is a --user option for docker run to only arbitrary non-root user other than mysql to start the db. Are there any similar solution for kubernetes env?

I tried to build my own docker image on top of mariadb like:

` from mariadb

RUN chmod -R 777 /var/lib/mysql EXPOSE3306 ` but this doesn't work. The permission of /var/lib/mysql still restrict to mysql:mysql.

[wglambert]

https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod

apiVersion: v1
kind: Pod
metadata:
  name: mariadb
spec:
  securityContext:
    runAsUser: 1337
    runAsGroup: 1337
    fsGroup: 2000
  volumes:
  - name: mariadb-volume
    emptyDir: {}
  containers:
  - name: mariadb
    image: mariadb
    env:
    - name: MYSQL_ROOT_PASSWORD
      value: root
    volumeMounts:
    - name: mariadb-volume
      mountPath: /var/lib/mysql
    securityContext:
      allowPrivilegeEscalation: false

$ kubectl apply -f mariadb.yaml
pod/mariadb created

$ kubectl logs mariadb
. . .
2020-11-04  0:45:21 0 [Note] mysqld: ready for connections.
Version: '10.5.6-MariaDB-1:10.5.6+maria~focal'  socket: '/run/mysqld/mysqld.sock'  port: 0  mariadb.org binary distribution

$ kubectl exec -it mariadb bash
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl kubectl exec [POD] -- [COMMAND] instead.
groups: cannot find name for group ID 1337
groups: cannot find name for group ID 2000

I have no name!@mariadb:/$  mysql -u root -proot
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 4
Server version: 10.5.6-MariaDB-1:10.5.6+maria~focal mariadb.org binary distribution

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]>

[zelong430]

Thanks wglambert. I did tried the empty dir approach. The /lib/var/mysql dir is open now which is good. But mysqld startup still returns an error below. Looks like some files are not found. Is that caused by I replace the docker /lib/var/mysql with the empty dir mount? do I need to manually copy any files?

2020-11-04  2:37:08 0 [Note] mysqld (mysqld 10.5.6-MariaDB-1:10.5.6+maria~focal) starting as process 27 ...
2020-11-04  2:37:08 0 [Warning] The parameter innodb_buffer_pool_instances is deprecated and has no effect.
2020-11-04  2:37:08 0 [Warning] The parameter innodb_thread_concurrency is deprecated and has no effect.
2020-11-04  2:37:08 0 [Note] InnoDB: Using Linux native AIO
2020-11-04  2:37:08 0 [Note] InnoDB: The first innodb_system data file 'ibdata1' did not exist. A new tablespace will be created!
2020-11-04  2:37:08 0 [Note] InnoDB: Uses event mutexes
2020-11-04  2:37:08 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2020-11-04  2:37:08 0 [Note] InnoDB: Number of pools: 1
2020-11-04  2:37:08 0 [Note] InnoDB: Using SSE4.2 crc32 instructions
2020-11-04  2:37:08 0 [Note] mysqld: O_TMPFILE is not supported on /tmp (disabling future attempts)
2020-11-04  2:37:08 0 [Note] InnoDB: Initializing buffer pool, total size = 4294967296, chunk size = 134217728
2020-11-04  2:37:08 0 [Note] InnoDB: Completed initialization of buffer pool
2020-11-04  2:37:08 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the
man page of setpriority().
2020-11-04  2:37:08 0 [Note] InnoDB: Setting file './ibdata1' size to 12 MB. Physically writing the file full; Please wait ...
2020-11-04  2:37:08 0 [Note] InnoDB: File './ibdata1' size is now 12 MB.
2020-11-04  2:37:08 0 [Note] InnoDB: Setting log file ./ib_logfile101 size to 52428800 bytes
2020-11-04  2:37:08 0 [Note] InnoDB: Renaming log file ./ib_logfile101 to ./ib_logfile0
2020-11-04  2:37:08 0 [Note] InnoDB: New log file created, LSN=10314
2020-11-04  2:37:08 0 [Note] InnoDB: Doublewrite buffer not found: creating new
2020-11-04  2:37:08 0 [Note] InnoDB: Doublewrite buffer created
2020-11-04  2:37:08 0 [Note] InnoDB: 128 rollback segments are active.
2020-11-04  2:37:08 0 [Note] InnoDB: Creating foreign key constraint system tables.
2020-11-04  2:37:08 0 [Note] InnoDB: Creating tablespace and datafile system tables.
2020-11-04  2:37:08 0 [Note] InnoDB: Creating sys_virtual system tables.
2020-11-04  2:37:08 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2020-11-04  2:37:08 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2020-11-04  2:37:08 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2020-11-04  2:37:08 0 [Note] InnoDB: 10.5.6 started; log sequence number 0; transaction id 7
2020-11-04  2:37:08 0 [Note] Plugin 'FEEDBACK' is disabled.
2020-11-04  2:37:08 0 [ERROR] Could not open mysql.plugin table: "Table 'mysql.plugin' doesn't exist". Some plugins may be not loaded
2020-11-04  2:37:08 0 [ERROR] Can't open and lock privilege tables: Table 'mysql.servers' doesn't exist
2020-11-04  2:37:08 0 [Warning] Failed to create a socket for IPv6 '::': errno: 97.
2020-11-04  2:37:08 0 [Note] Server socket created on IP: '0.0.0.0'.
2020-11-04  2:37:08 0 [ERROR] Fatal error: Can't open and lock privilege tables: Table 'mysql.db' doesn't exist
2020-11-04  2:37:08 0 [ERROR] Aborting

[wglambert]

Can you post the yaml you're using, it almost seems like some sort of database corruption but if an empty directory is being initialized then that doesn't make sense.

You could also try asking over at the Docker Community Forums, Docker Community Slack, or Stack Overflow. Since these repos aren't really a user-help forum

[wglambert]

Going to close assuming you've resolved this

[mirekphd]

This is how OKD / Openshift works by default... what's more, you cannot guess the UID, as it is randomly selected from a wide range of permitted UIDs for each user (i.e. her namespace / project).

The challenge is that our Kubernetes cluster only allow me to deploy pods with a fixed default security context (a fixed specific UID/GID). Therefore, I can't run the pod as root or as any users other than the specific UID/GID allowed

[mirekphd]

Note that a config like this would fail to persist database outside of the container and is thus a recipe for quick and total data loss imho... turns it into a poor-woman's in-memory database - one without any periodic permanent snapshots...

  volumes:
  - name: mariadb-volume
    emptyDir: {}