cannot create directory '/var/lib/mysql/mysql': Permission denied

[mayask]

Hi

I'm trying to run a galera-cluster on Kubernetes. It works pretty ok, by the way ;)

When using official mariadb image, container successfully initializes /var/lib/mysql directory mounted from host machine with the following (default) permissions:

[root@localhost docker-multinode]# ll /data/
total 0
drwxr-xr-x. 2 root root 40 Jan 18 11:10 mysql

However, when using galera cluster setup from this image, it fails to initialize mysql_data dir due to permission errors, even though volume setup/host permissions didn't change.

mysqld.sh: Initializing MariaDb data directory...
mkdir: cannot create directory '/var/lib/mysql/mysql': Permission denied
Fatal error Can't create database directory '/var/lib/mysql/mysql'

The latest information about mysql_install_db is available at
https://mariadb.com/kb/en/installing-system-tables-mysql_install_db
mysqld.sh: Failed to initialized data directory. Will hope for the best...
mysqld.sh: ----------------------------------
mysqld.sh: Starting with options: --console --init-file=/tmp/bootstrap.sql --wsrep-on=ON --wsrep-new-cluster --wsrep_cluster_name=dbcluster --wsrep_cluster_address=gcomm:// --wsrep_node_address=10.
1.7.2:4567 --wsrep_sst_auth=xtrabackup:123 --default-time-zone=+00:00 
2017-01-18 11:10:05 140254827345856 [Note] mysqld (mysqld 10.1.19-MariaDB-1~jessie) starting as process 22 ...
2017-01-18 11:10:05 140254827345856 [Warning] Can't create test file /var/lib/mysql/galera-seed.lower-test
2017-01-18 11:10:05 140254827345856 [Note] WSREP: Read nil XID from storage engines, skipping position init
2017-01-18 11:10:05 140254827345856 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/galera/libgalera_smm.so'
2017-01-18 11:10:05 140254827345856 [Note] WSREP: wsrep_load(): Galera 25.3.18(r3632) by Codership Oy <info@codership.com> loaded successfully.
2017-01-18 11:10:05 140254827345856 [Note] WSREP: CRC-32C: using hardware acceleration.
2017-01-18 11:10:05 140254827345856 [Warning] WSREP: Could not open state file for reading: '/var/lib/mysql//grastate.dat'
2017-01-18 11:10:05 140254827345856 [ERROR] WSREP: Could not open state file for writing: '/var/lib/mysql//grastate.dat'. Check permissions and/or disk space.: 13 (Permission denied)
         at galera/src/saved_state.cpp:SavedState():49
2017-01-18 11:10:05 140254827345856 [ERROR] WSREP: wsrep::init() failed: 7, must shutdown
2017-01-18 11:10:05 140254827345856 [ERROR] Aborting

Could you please give me a hint on what could I do in this case?

This happens here https://github.com/colinmollenhour/mariadb-galera-swarm/blob/master/mysqld.sh#L13

[mayask]

Setting chmod -R 777 /var/lib/mysql on host does solve the problem. Going back to chmod -R 775 /var/lib/mysql on host bring up the issue again.

[colinmollenhour]

Hi Maxim, unfortunately I have no experience with Kubernetes so I don't know if the problem is specific to Kubernetes or not but I have not had that problem on Docker Swarm or Kontena although I have been using named volumes so that might be the difference. The mysqld.sh script is run as "mysql" user using gosu, so try adding a chown to start.sh while the user is still root. I think chown is appropriate rather than chmod 777 for security reasons.

[mayask]

Thanks for the answer! Will try and get back to you

[mayask]

The question is not specific to Kubernetes, since Docker is still backing it up.

[colinmollenhour]

I added a "chown" in the start.sh script to make sure the directory is owned by mysql. I don't know if this will fix your issue or not but I don't know what else would. Thanks!