search

docker / for-linux

Docker Engine for Linux
https://docs.docker.com/engine/installation/
757 stars 86 forks source link

MySQL does not start with overlay2 and overlay but starts with aufs #72

Open aalexgabi opened 7 years ago

aalexgabi commented 7 years ago

Expected behavior

MySQL should start

Actual behavior

MySQL does not start and returns an error:

root@bff6f2f06129:/# mysqld
170801 15:19:27 [Warning] Using unique option prefix key_buffer instead of key_buffer_size is deprecated and will be removed in a future release. Please use the full name instead.
170801 15:19:27 [Note] mysqld (mysqld 5.5.57-0+deb7u1) starting as process 6 ...
170801 15:19:27 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead.
170801 15:19:27 [Note] Plugin 'FEDERATED' is disabled.
mysqld: Got error 140 from storage engine
170801 15:19:27 [ERROR] Can't open the mysql.plugin table. Please run mysql_upgrade to create it.
170801 15:19:27 InnoDB: The InnoDB memory heap is disabled
170801 15:19:27 InnoDB: Mutexes and rw_locks use GCC atomic builtins
170801 15:19:27 InnoDB: Compressed tables use zlib 1.2.7
170801 15:19:27 InnoDB: Using Linux native AIO
170801 15:19:27 InnoDB: Initializing buffer pool, size = 128.0M
170801 15:19:27 InnoDB: Completed initialization of buffer pool
170801 15:19:27 InnoDB: highest supported file format is Barracuda.
170801 15:19:27  InnoDB: Waiting for the background threads to start
170801 15:19:28 InnoDB: 5.5.57 started; log sequence number 1595685
170801 15:19:28 [Note] Server hostname (bind-address): '127.0.0.1'; port: 3306
170801 15:19:28 [Note]   - '127.0.0.1' resolves to '127.0.0.1';
170801 15:19:28 [Note] Server socket created on IP: '127.0.0.1'.
170801 15:19:28 [ERROR] Fatal error: Can't open and lock privilege tables: Got error 140 from storage engine

Steps to reproduce the behavior

Using Dockerfile:

FROM debian:wheezy

RUN apt-get update && apt-get install -y mysql-server-5.5
RUN service mysql start

Output:

root@thor:~/mysql-basic-test# docker build .
Sending build context to Docker daemon  2.048kB
Step 1/3 : FROM debian:wheezy
 ---> c72c50c45f17
Step 2/3 : RUN apt-get update && apt-get install -y mysql-server-5.5
 ---> Using cache
 ---> 692cd7218986
Step 3/3 : RUN service mysql start
 ---> Running in fedd9ef25d69
Starting MySQL database server: mysqld . . . . . . . . . . . . . . failed!
The command '/bin/sh -c service mysql start' returned a non-zero code: 1
root@thor:~/mysql-basic-test# docker build --no-cache .
Sending build context to Docker daemon  2.048kB
Step 1/3 : FROM debian:wheezy
 ---> c72c50c45f17
Step 2/3 : RUN apt-get update && apt-get install -y mysql-server-5.5
 ---> Running in 9accc804f9ac
Get:1 http://security.debian.org wheezy/updates Release.gpg [1554 B]
Get:2 http://security.debian.org wheezy/updates Release [39.0 kB]
Get:3 http://security.debian.org wheezy/updates/main amd64 Packages [697 kB]
Get:4 http://deb.debian.org wheezy Release.gpg [2373 B]
Get:5 http://deb.debian.org wheezy-updates Release.gpg [1554 B]
Get:6 http://deb.debian.org wheezy Release [191 kB]
Get:7 http://deb.debian.org wheezy-updates Release [155 kB]
Get:8 http://deb.debian.org wheezy/main amd64 Packages [7634 kB]
Get:9 http://deb.debian.org wheezy-updates/main amd64 Packages [7481 B]
Fetched 8729 kB in 2s (3979 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
The following extra packages will be installed:
  adduser heirloom-mailx krb5-locales libaio1 libclass-isa-perl
  libdbd-mysql-perl libdbi-perl libgdbm3 libgssapi-krb5-2
  libhtml-template-perl libk5crypto3 libkeyutils1 libkrb5-3 libkrb5support0
  libmysqlclient18 libssl1.0.0 libswitch-perl libwrap0 mysql-client-5.5
  mysql-common mysql-server-core-5.5 perl perl-modules psmisc tcpd
Suggested packages:
  exim4 mail-transport-agent krb5-doc krb5-user libipc-sharedcache-perl
  libterm-readkey-perl tinyca perl-doc libterm-readline-gnu-perl
  libterm-readline-perl-perl make libpod-plainer-perl
Recommended packages:
  mailx
The following NEW packages will be installed:
  adduser heirloom-mailx krb5-locales libaio1 libclass-isa-perl
  libdbd-mysql-perl libdbi-perl libgdbm3 libgssapi-krb5-2
  libhtml-template-perl libk5crypto3 libkeyutils1 libkrb5-3 libkrb5support0
  libmysqlclient18 libssl1.0.0 libswitch-perl libwrap0 mysql-client-5.5
  mysql-common mysql-server-5.5 mysql-server-core-5.5 perl perl-modules psmisc
  tcpd
0 upgraded, 26 newly installed, 0 to remove and 0 not upgraded.
Need to get 21.8 MB of archives.
After this operation, 139 MB of additional disk space will be used.
Get:1 http://security.debian.org/ wheezy/updates/main libssl1.0.0 amd64 1.0.1t-1+deb7u2 [1278 kB]
Get:2 http://security.debian.org/ wheezy/updates/main mysql-common all 5.5.57-0+deb7u1 [83.9 kB]
Get:3 http://security.debian.org/ wheezy/updates/main libmysqlclient18 amd64 5.5.57-0+deb7u1 [691 kB]
Get:4 http://deb.debian.org/debian/ wheezy/main libgdbm3 amd64 1.8.3-11 [46.9 kB]
Get:5 http://deb.debian.org/debian/ wheezy/main libkeyutils1 amd64 1.5.5-3+deb7u1 [8664 B]
Get:6 http://security.debian.org/ wheezy/updates/main perl-modules all 5.14.2-21+deb7u5 [3441 kB]
Get:7 http://deb.debian.org/debian/ wheezy/main libkrb5support0 amd64 1.10.1+dfsg-5+deb7u7 [50.5 kB]
Get:8 http://deb.debian.org/debian/ wheezy/main libk5crypto3 amd64 1.10.1+dfsg-5+deb7u7 [113 kB]
Get:9 http://deb.debian.org/debian/ wheezy/main libkrb5-3 amd64 1.10.1+dfsg-5+deb7u7 [394 kB]
Get:10 http://security.debian.org/ wheezy/updates/main perl amd64 5.14.2-21+deb7u5 [4434 kB]
Get:11 http://deb.debian.org/debian/ wheezy/main libgssapi-krb5-2 amd64 1.10.1+dfsg-5+deb7u7 [150 kB]
Get:12 http://deb.debian.org/debian/ wheezy/main libwrap0 amd64 7.6.q-24 [62.4 kB]
Get:13 http://deb.debian.org/debian/ wheezy/main libaio1 amd64 0.3.109-3 [9150 B]
Get:14 http://deb.debian.org/debian/ wheezy/main adduser all 3.113+nmu3 [264 kB]
Get:15 http://deb.debian.org/debian/ wheezy/main libswitch-perl all 2.16-2 [21.0 kB]
Get:16 http://deb.debian.org/debian/ wheezy/main libclass-isa-perl all 0.36-3 [12.3 kB]
Get:17 http://security.debian.org/ wheezy/updates/main libdbd-mysql-perl amd64 4.021-1+deb7u2 [126 kB]
Get:18 http://deb.debian.org/debian/ wheezy/main libdbi-perl amd64 1.622-1+deb7u1 [892 kB]
Get:19 http://security.debian.org/ wheezy/updates/main mysql-client-5.5 amd64 5.5.57-0+deb7u1 [1778 kB]
Get:20 http://deb.debian.org/debian/ wheezy/main psmisc amd64 22.19-1+deb7u1 [135 kB]
Get:21 http://deb.debian.org/debian/ wheezy/main krb5-locales all 1.10.1+dfsg-5+deb7u7 [1504 kB]
Get:22 http://deb.debian.org/debian/ wheezy/main heirloom-mailx amd64 12.5-2+deb7u1 [274 kB]
Get:23 http://deb.debian.org/debian/ wheezy/main libhtml-template-perl all 2.91-1 [72.0 kB]
Get:24 http://deb.debian.org/debian/ wheezy/main tcpd amd64 7.6.q-24 [28.2 kB]
Get:25 http://security.debian.org/ wheezy/updates/main mysql-server-core-5.5 amd64 5.5.57-0+deb7u1 [3980 kB]
Get:26 http://security.debian.org/ wheezy/updates/main mysql-server-5.5 amd64 5.5.57-0+deb7u1 [1967 kB]
debconf: delaying package configuration, since apt-utils is not installed
Fetched 21.8 MB in 6s (3518 kB/s)
Selecting previously unselected package libgdbm3:amd64.
(Reading database ... 6747 files and directories currently installed.)
Unpacking libgdbm3:amd64 (from .../libgdbm3_1.8.3-11_amd64.deb) ...
Selecting previously unselected package libssl1.0.0:amd64.
Unpacking libssl1.0.0:amd64 (from .../libssl1.0.0_1.0.1t-1+deb7u2_amd64.deb) ...
Selecting previously unselected package libkeyutils1:amd64.
Unpacking libkeyutils1:amd64 (from .../libkeyutils1_1.5.5-3+deb7u1_amd64.deb) ...
Selecting previously unselected package libkrb5support0:amd64.
Unpacking libkrb5support0:amd64 (from .../libkrb5support0_1.10.1+dfsg-5+deb7u7_amd64.deb) ...
Selecting previously unselected package libk5crypto3:amd64.
Unpacking libk5crypto3:amd64 (from .../libk5crypto3_1.10.1+dfsg-5+deb7u7_amd64.deb) ...
Selecting previously unselected package libkrb5-3:amd64.
Unpacking libkrb5-3:amd64 (from .../libkrb5-3_1.10.1+dfsg-5+deb7u7_amd64.deb) ...
Selecting previously unselected package libgssapi-krb5-2:amd64.
Unpacking libgssapi-krb5-2:amd64 (from .../libgssapi-krb5-2_1.10.1+dfsg-5+deb7u7_amd64.deb) ...
Selecting previously unselected package libwrap0:amd64.
Unpacking libwrap0:amd64 (from .../libwrap0_7.6.q-24_amd64.deb) ...
Selecting previously unselected package libaio1:amd64.
Unpacking libaio1:amd64 (from .../libaio1_0.3.109-3_amd64.deb) ...
Selecting previously unselected package mysql-common.
Unpacking mysql-common (from .../mysql-common_5.5.57-0+deb7u1_all.deb) ...
Selecting previously unselected package libmysqlclient18:amd64.
Unpacking libmysqlclient18:amd64 (from .../libmysqlclient18_5.5.57-0+deb7u1_amd64.deb) ...
Selecting previously unselected package adduser.
Unpacking adduser (from .../adduser_3.113+nmu3_all.deb) ...
Selecting previously unselected package libswitch-perl.
Unpacking libswitch-perl (from .../libswitch-perl_2.16-2_all.deb) ...
Selecting previously unselected package libclass-isa-perl.
Unpacking libclass-isa-perl (from .../libclass-isa-perl_0.36-3_all.deb) ...
Selecting previously unselected package perl-modules.
Unpacking perl-modules (from .../perl-modules_5.14.2-21+deb7u5_all.deb) ...
Selecting previously unselected package perl.
Unpacking perl (from .../perl_5.14.2-21+deb7u5_amd64.deb) ...
Selecting previously unselected package libdbi-perl.
Unpacking libdbi-perl (from .../libdbi-perl_1.622-1+deb7u1_amd64.deb) ...
Selecting previously unselected package libdbd-mysql-perl.
Unpacking libdbd-mysql-perl (from .../libdbd-mysql-perl_4.021-1+deb7u2_amd64.deb) ...
Selecting previously unselected package mysql-client-5.5.
Unpacking mysql-client-5.5 (from .../mysql-client-5.5_5.5.57-0+deb7u1_amd64.deb) ...
Selecting previously unselected package psmisc.
Unpacking psmisc (from .../psmisc_22.19-1+deb7u1_amd64.deb) ...
Selecting previously unselected package mysql-server-core-5.5.
Unpacking mysql-server-core-5.5 (from .../mysql-server-core-5.5_5.5.57-0+deb7u1_amd64.deb) ...
Setting up mysql-common (5.5.57-0+deb7u1) ...
Setting up adduser (3.113+nmu3) ...
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (This frontend requires a controlling tty.)
debconf: falling back to frontend: Teletype
Selecting previously unselected package mysql-server-5.5.
(Reading database ... 8756 files and directories currently installed.)
Unpacking mysql-server-5.5 (from .../mysql-server-5.5_5.5.57-0+deb7u1_amd64.deb) ...
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (This frontend requires a controlling tty.)
debconf: falling back to frontend: Teletype
Configuring mysql-server-5.5
----------------------------

While not mandatory, it is highly recommended that you set a password for the 
MySQL administrative "root" user.

If this field is left blank, the password will not be changed.

New password for the MySQL "root" user: 
Use of uninitialized value $_[1] in join or string at /usr/share/perl5/Debconf/DbDriver/Stack.pm line 111.
Use of uninitialized value $val in substitution (s///) at /usr/share/perl5/Debconf/Format/822.pm line 83, <GEN6> line 1.
Use of uninitialized value $val in concatenation (.) or string at /usr/share/perl5/Debconf/Format/822.pm line 84, <GEN6> line 1.
Selecting previously unselected package krb5-locales.
Unpacking krb5-locales (from .../krb5-locales_1.10.1+dfsg-5+deb7u7_all.deb) ...
Selecting previously unselected package heirloom-mailx.
Unpacking heirloom-mailx (from .../heirloom-mailx_12.5-2+deb7u1_amd64.deb) ...
Selecting previously unselected package libhtml-template-perl.
Unpacking libhtml-template-perl (from .../libhtml-template-perl_2.91-1_all.deb) ...
Selecting previously unselected package tcpd.
Unpacking tcpd (from .../tcpd_7.6.q-24_amd64.deb) ...
Setting up libgdbm3:amd64 (1.8.3-11) ...
Setting up libssl1.0.0:amd64 (1.0.1t-1+deb7u2) ...
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (This frontend requires a controlling tty.)
debconf: falling back to frontend: Teletype
Setting up libkeyutils1:amd64 (1.5.5-3+deb7u1) ...
Setting up libkrb5support0:amd64 (1.10.1+dfsg-5+deb7u7) ...
Setting up libk5crypto3:amd64 (1.10.1+dfsg-5+deb7u7) ...
Setting up libkrb5-3:amd64 (1.10.1+dfsg-5+deb7u7) ...
Setting up libgssapi-krb5-2:amd64 (1.10.1+dfsg-5+deb7u7) ...
Setting up libwrap0:amd64 (7.6.q-24) ...
Setting up libaio1:amd64 (0.3.109-3) ...
Setting up libmysqlclient18:amd64 (5.5.57-0+deb7u1) ...
Setting up libclass-isa-perl (0.36-3) ...
Setting up psmisc (22.19-1+deb7u1) ...
Setting up mysql-server-core-5.5 (5.5.57-0+deb7u1) ...
Setting up krb5-locales (1.10.1+dfsg-5+deb7u7) ...
Setting up heirloom-mailx (12.5-2+deb7u1) ...
update-alternatives: using /usr/bin/heirloom-mailx to provide /usr/bin/mailx (mailx) in auto mode
Setting up tcpd (7.6.q-24) ...
Setting up libswitch-perl (2.16-2) ...
Setting up perl-modules (5.14.2-21+deb7u5) ...
Setting up perl (5.14.2-21+deb7u5) ...
update-alternatives: using /usr/bin/prename to provide /usr/bin/rename (rename) in auto mode
Setting up libdbi-perl (1.622-1+deb7u1) ...
Setting up libdbd-mysql-perl (4.021-1+deb7u2) ...
Setting up mysql-client-5.5 (5.5.57-0+deb7u1) ...
Setting up mysql-server-5.5 (5.5.57-0+deb7u1) ...
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (This frontend requires a controlling tty.)
debconf: falling back to frontend: Teletype
Configuring mysql-server-5.5
----------------------------

While not mandatory, it is highly recommended that you set a password for the 
MySQL administrative "root" user.

If this field is left blank, the password will not be changed.

New password for the MySQL "root" user: 
Use of uninitialized value $_[1] in join or string at /usr/share/perl5/Debconf/DbDriver/Stack.pm line 111.
invoke-rc.d: policy-rc.d denied execution of stop.
invoke-rc.d: policy-rc.d denied execution of start.
Setting up libhtml-template-perl (2.91-1) ...
 ---> 4001a489928f
Removing intermediate container 9accc804f9ac
Step 3/3 : RUN service mysql start
 ---> Running in 7c5832ab6216
Starting MySQL database server: mysqld . . . . . . . . . . . . . . failed!
The command '/bin/sh -c service mysql start' returned a non-zero code: 1
root@thor:~/mysql-basic-test#

Interestingly MySQL starts when is started in the same intermediate container: Using Dockerfile:

FROM debian:wheezy

RUN apt-get update && apt-get install -y mysql-server-5.5 && service mysql start && sleep 5 && service mysql stop && sleep 5
RUN service mysql start

Output:

root@thor:~/mysql-basic-test# docker build --no-cache .
Sending build context to Docker daemon  2.048kB
Step 1/3 : FROM debian:wheezy
 ---> c72c50c45f17
Step 2/3 : RUN apt-get update && apt-get install -y mysql-server-5.5 && service mysql start && sleep 5 && service mysql stop && sleep 5
 ---> Running in 95b404fa13f0
Get:1 http://security.debian.org wheezy/updates Release.gpg [1554 B]
Get:2 http://security.debian.org wheezy/updates Release [39.0 kB]
Get:3 http://security.debian.org wheezy/updates/main amd64 Packages [697 kB]
Get:4 http://deb.debian.org wheezy Release.gpg [2373 B]
Get:5 http://deb.debian.org wheezy-updates Release.gpg [1554 B]
Get:6 http://deb.debian.org wheezy Release [191 kB]
Get:7 http://deb.debian.org wheezy-updates Release [155 kB]
Get:8 http://deb.debian.org wheezy/main amd64 Packages [7634 kB]
Get:9 http://deb.debian.org wheezy-updates/main amd64 Packages [7481 B]
Fetched 8729 kB in 2s (4035 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
The following extra packages will be installed:
  adduser heirloom-mailx krb5-locales libaio1 libclass-isa-perl
  libdbd-mysql-perl libdbi-perl libgdbm3 libgssapi-krb5-2
  libhtml-template-perl libk5crypto3 libkeyutils1 libkrb5-3 libkrb5support0
  libmysqlclient18 libssl1.0.0 libswitch-perl libwrap0 mysql-client-5.5
  mysql-common mysql-server-core-5.5 perl perl-modules psmisc tcpd
Suggested packages:
  exim4 mail-transport-agent krb5-doc krb5-user libipc-sharedcache-perl
  libterm-readkey-perl tinyca perl-doc libterm-readline-gnu-perl
  libterm-readline-perl-perl make libpod-plainer-perl
Recommended packages:
  mailx
The following NEW packages will be installed:
  adduser heirloom-mailx krb5-locales libaio1 libclass-isa-perl
  libdbd-mysql-perl libdbi-perl libgdbm3 libgssapi-krb5-2
  libhtml-template-perl libk5crypto3 libkeyutils1 libkrb5-3 libkrb5support0
  libmysqlclient18 libssl1.0.0 libswitch-perl libwrap0 mysql-client-5.5
  mysql-common mysql-server-5.5 mysql-server-core-5.5 perl perl-modules psmisc
  tcpd
0 upgraded, 26 newly installed, 0 to remove and 0 not upgraded.
Need to get 21.8 MB of archives.
After this operation, 139 MB of additional disk space will be used.
Get:1 http://security.debian.org/ wheezy/updates/main libssl1.0.0 amd64 1.0.1t-1+deb7u2 [1278 kB]
Get:2 http://deb.debian.org/debian/ wheezy/main libgdbm3 amd64 1.8.3-11 [46.9 kB]
Get:3 http://security.debian.org/ wheezy/updates/main mysql-common all 5.5.57-0+deb7u1 [83.9 kB]
Get:4 http://deb.debian.org/debian/ wheezy/main libkeyutils1 amd64 1.5.5-3+deb7u1 [8664 B]
Get:5 http://security.debian.org/ wheezy/updates/main libmysqlclient18 amd64 5.5.57-0+deb7u1 [691 kB]
Get:6 http://deb.debian.org/debian/ wheezy/main libkrb5support0 amd64 1.10.1+dfsg-5+deb7u7 [50.5 kB]
Get:7 http://security.debian.org/ wheezy/updates/main perl-modules all 5.14.2-21+deb7u5 [3441 kB]
Get:8 http://deb.debian.org/debian/ wheezy/main libk5crypto3 amd64 1.10.1+dfsg-5+deb7u7 [113 kB]
Get:9 http://deb.debian.org/debian/ wheezy/main libkrb5-3 amd64 1.10.1+dfsg-5+deb7u7 [394 kB]
Get:10 http://deb.debian.org/debian/ wheezy/main libgssapi-krb5-2 amd64 1.10.1+dfsg-5+deb7u7 [150 kB]
Get:11 http://deb.debian.org/debian/ wheezy/main libwrap0 amd64 7.6.q-24 [62.4 kB]
Get:12 http://deb.debian.org/debian/ wheezy/main libaio1 amd64 0.3.109-3 [9150 B]
Get:13 http://deb.debian.org/debian/ wheezy/main adduser all 3.113+nmu3 [264 kB]
Get:14 http://deb.debian.org/debian/ wheezy/main libswitch-perl all 2.16-2 [21.0 kB]
Get:15 http://deb.debian.org/debian/ wheezy/main libclass-isa-perl all 0.36-3 [12.3 kB]
Get:16 http://security.debian.org/ wheezy/updates/main perl amd64 5.14.2-21+deb7u5 [4434 kB]
Get:17 http://deb.debian.org/debian/ wheezy/main libdbi-perl amd64 1.622-1+deb7u1 [892 kB]
Get:18 http://deb.debian.org/debian/ wheezy/main psmisc amd64 22.19-1+deb7u1 [135 kB]
Get:19 http://deb.debian.org/debian/ wheezy/main krb5-locales all 1.10.1+dfsg-5+deb7u7 [1504 kB]
Get:20 http://deb.debian.org/debian/ wheezy/main heirloom-mailx amd64 12.5-2+deb7u1 [274 kB]
Get:21 http://deb.debian.org/debian/ wheezy/main libhtml-template-perl all 2.91-1 [72.0 kB]
Get:22 http://deb.debian.org/debian/ wheezy/main tcpd amd64 7.6.q-24 [28.2 kB]
Get:23 http://security.debian.org/ wheezy/updates/main libdbd-mysql-perl amd64 4.021-1+deb7u2 [126 kB]
Get:24 http://security.debian.org/ wheezy/updates/main mysql-client-5.5 amd64 5.5.57-0+deb7u1 [1778 kB]
Get:25 http://security.debian.org/ wheezy/updates/main mysql-server-core-5.5 amd64 5.5.57-0+deb7u1 [3980 kB]
Get:26 http://security.debian.org/ wheezy/updates/main mysql-server-5.5 amd64 5.5.57-0+deb7u1 [1967 kB]
debconf: delaying package configuration, since apt-utils is not installed
Fetched 21.8 MB in 3s (5851 kB/s)
Selecting previously unselected package libgdbm3:amd64.
(Reading database ... 6747 files and directories currently installed.)
Unpacking libgdbm3:amd64 (from .../libgdbm3_1.8.3-11_amd64.deb) ...
Selecting previously unselected package libssl1.0.0:amd64.
Unpacking libssl1.0.0:amd64 (from .../libssl1.0.0_1.0.1t-1+deb7u2_amd64.deb) ...
Selecting previously unselected package libkeyutils1:amd64.
Unpacking libkeyutils1:amd64 (from .../libkeyutils1_1.5.5-3+deb7u1_amd64.deb) ...
Selecting previously unselected package libkrb5support0:amd64.
Unpacking libkrb5support0:amd64 (from .../libkrb5support0_1.10.1+dfsg-5+deb7u7_amd64.deb) ...
Selecting previously unselected package libk5crypto3:amd64.
Unpacking libk5crypto3:amd64 (from .../libk5crypto3_1.10.1+dfsg-5+deb7u7_amd64.deb) ...
Selecting previously unselected package libkrb5-3:amd64.
Unpacking libkrb5-3:amd64 (from .../libkrb5-3_1.10.1+dfsg-5+deb7u7_amd64.deb) ...
Selecting previously unselected package libgssapi-krb5-2:amd64.
Unpacking libgssapi-krb5-2:amd64 (from .../libgssapi-krb5-2_1.10.1+dfsg-5+deb7u7_amd64.deb) ...
Selecting previously unselected package libwrap0:amd64.
Unpacking libwrap0:amd64 (from .../libwrap0_7.6.q-24_amd64.deb) ...
Selecting previously unselected package libaio1:amd64.
Unpacking libaio1:amd64 (from .../libaio1_0.3.109-3_amd64.deb) ...
Selecting previously unselected package mysql-common.
Unpacking mysql-common (from .../mysql-common_5.5.57-0+deb7u1_all.deb) ...
Selecting previously unselected package libmysqlclient18:amd64.
Unpacking libmysqlclient18:amd64 (from .../libmysqlclient18_5.5.57-0+deb7u1_amd64.deb) ...
Selecting previously unselected package adduser.
Unpacking adduser (from .../adduser_3.113+nmu3_all.deb) ...
Selecting previously unselected package libswitch-perl.
Unpacking libswitch-perl (from .../libswitch-perl_2.16-2_all.deb) ...
Selecting previously unselected package libclass-isa-perl.
Unpacking libclass-isa-perl (from .../libclass-isa-perl_0.36-3_all.deb) ...
Selecting previously unselected package perl-modules.
Unpacking perl-modules (from .../perl-modules_5.14.2-21+deb7u5_all.deb) ...
Selecting previously unselected package perl.
Unpacking perl (from .../perl_5.14.2-21+deb7u5_amd64.deb) ...
Selecting previously unselected package libdbi-perl.
Unpacking libdbi-perl (from .../libdbi-perl_1.622-1+deb7u1_amd64.deb) ...
Selecting previously unselected package libdbd-mysql-perl.
Unpacking libdbd-mysql-perl (from .../libdbd-mysql-perl_4.021-1+deb7u2_amd64.deb) ...
Selecting previously unselected package mysql-client-5.5.
Unpacking mysql-client-5.5 (from .../mysql-client-5.5_5.5.57-0+deb7u1_amd64.deb) ...
Selecting previously unselected package psmisc.
Unpacking psmisc (from .../psmisc_22.19-1+deb7u1_amd64.deb) ...
Selecting previously unselected package mysql-server-core-5.5.
Unpacking mysql-server-core-5.5 (from .../mysql-server-core-5.5_5.5.57-0+deb7u1_amd64.deb) ...
Setting up mysql-common (5.5.57-0+deb7u1) ...
Setting up adduser (3.113+nmu3) ...
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (This frontend requires a controlling tty.)
debconf: falling back to frontend: Teletype
Selecting previously unselected package mysql-server-5.5.
(Reading database ... 8756 files and directories currently installed.)
Unpacking mysql-server-5.5 (from .../mysql-server-5.5_5.5.57-0+deb7u1_amd64.deb) ...
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (This frontend requires a controlling tty.)
debconf: falling back to frontend: Teletype
Configuring mysql-server-5.5
----------------------------

While not mandatory, it is highly recommended that you set a password for the 
MySQL administrative "root" user.

If this field is left blank, the password will not be changed.

New password for the MySQL "root" user: 
Use of uninitialized value $_[1] in join or string at /usr/share/perl5/Debconf/DbDriver/Stack.pm line 111.
Use of uninitialized value $val in substitution (s///) at /usr/share/perl5/Debconf/Format/822.pm line 83, <GEN6> line 1.
Use of uninitialized value $val in concatenation (.) or string at /usr/share/perl5/Debconf/Format/822.pm line 84, <GEN6> line 1.
Selecting previously unselected package krb5-locales.
Unpacking krb5-locales (from .../krb5-locales_1.10.1+dfsg-5+deb7u7_all.deb) ...
Selecting previously unselected package heirloom-mailx.
Unpacking heirloom-mailx (from .../heirloom-mailx_12.5-2+deb7u1_amd64.deb) ...
Selecting previously unselected package libhtml-template-perl.
Unpacking libhtml-template-perl (from .../libhtml-template-perl_2.91-1_all.deb) ...
Selecting previously unselected package tcpd.
Unpacking tcpd (from .../tcpd_7.6.q-24_amd64.deb) ...
Setting up libgdbm3:amd64 (1.8.3-11) ...
Setting up libssl1.0.0:amd64 (1.0.1t-1+deb7u2) ...
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (This frontend requires a controlling tty.)
debconf: falling back to frontend: Teletype
Setting up libkeyutils1:amd64 (1.5.5-3+deb7u1) ...
Setting up libkrb5support0:amd64 (1.10.1+dfsg-5+deb7u7) ...
Setting up libk5crypto3:amd64 (1.10.1+dfsg-5+deb7u7) ...
Setting up libkrb5-3:amd64 (1.10.1+dfsg-5+deb7u7) ...
Setting up libgssapi-krb5-2:amd64 (1.10.1+dfsg-5+deb7u7) ...
Setting up libwrap0:amd64 (7.6.q-24) ...
Setting up libaio1:amd64 (0.3.109-3) ...
Setting up libmysqlclient18:amd64 (5.5.57-0+deb7u1) ...
Setting up libclass-isa-perl (0.36-3) ...
Setting up psmisc (22.19-1+deb7u1) ...
Setting up mysql-server-core-5.5 (5.5.57-0+deb7u1) ...
Setting up krb5-locales (1.10.1+dfsg-5+deb7u7) ...
Setting up heirloom-mailx (12.5-2+deb7u1) ...
update-alternatives: using /usr/bin/heirloom-mailx to provide /usr/bin/mailx (mailx) in auto mode
Setting up tcpd (7.6.q-24) ...
Setting up libswitch-perl (2.16-2) ...
Setting up perl-modules (5.14.2-21+deb7u5) ...
Setting up perl (5.14.2-21+deb7u5) ...
update-alternatives: using /usr/bin/prename to provide /usr/bin/rename (rename) in auto mode
Setting up libdbi-perl (1.622-1+deb7u1) ...
Setting up libdbd-mysql-perl (4.021-1+deb7u2) ...
Setting up mysql-client-5.5 (5.5.57-0+deb7u1) ...
Setting up mysql-server-5.5 (5.5.57-0+deb7u1) ...
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (This frontend requires a controlling tty.)
debconf: falling back to frontend: Teletype
Configuring mysql-server-5.5
----------------------------

While not mandatory, it is highly recommended that you set a password for the 
MySQL administrative "root" user.

If this field is left blank, the password will not be changed.

New password for the MySQL "root" user: 
Use of uninitialized value $_[1] in join or string at /usr/share/perl5/Debconf/DbDriver/Stack.pm line 111.
invoke-rc.d: policy-rc.d denied execution of stop.
invoke-rc.d: policy-rc.d denied execution of start.
Setting up libhtml-template-perl (2.91-1) ...
Starting MySQL database server: mysqld ..
Checking for tables which need an upgrade, are corrupt or were 
not closed cleanly..
Stopping MySQL database server: mysqld.
 ---> 46a5157fd569
Removing intermediate container 95b404fa13f0
Step 3/3 : RUN service mysql start
 ---> Running in 064e2b4dd2e3
Starting MySQL database server: mysqld . . . . . . . . . . . . . . failed!
The command '/bin/sh -c service mysql start' returned a non-zero code: 1
root@thor:~/mysql-basic-test#

Output of docker version:

root@thor:~/mysql-basic-test# docker version
Client:
 Version:      17.05.0-ce
 API version:  1.29
 Go version:   go1.7.5
 Git commit:   89658be
 Built:        Thu May  4 22:04:27 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.05.0-ce
 API version:  1.29 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   89658be
 Built:        Thu May  4 22:04:27 2017
 OS/Arch:      linux/amd64
 Experimental: false

Output of docker info:

root@thor:~/mysql-basic-test# docker info
Containers: 21
 Running: 1
 Paused: 0
 Stopped: 20
Images: 1109
Server Version: 17.05.0-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins: 
 Volume: local
 Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 9048e5e50717ea4497b757314bad98ea3763c145
runc version: 9c2d8d184e5da67c95d601382adf14862e4f2228
init version: 949e6fa
Kernel Version: 4.9.0-0.bpo.3-amd64
Operating System: Debian GNU/Linux 8 (jessie)
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 7.763GiB
Name: thor
ID: IZTB:XN5O:KVZK:LOTA:NPTQ:L2EP:QGBJ:BW3Q:U3U6:6IMY:63WS:U3OF
Docker Root Dir: /home/tc/.docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
 10.144.112.139:5139
 127.0.0.0/8
Live Restore Enabled: false

WARNING: No swap limit support

Additional environment details (AWS, VirtualBox, physical, etc.)

I tried with the Linux 4.9 Kernel from Jessie backports and Linux 4.11 from Liquorix and the problem exists for both.

The problem arises when using overlay2 and overlay. The problem does not arise when using aufs.

aalexgabi commented 7 years ago

I could also reproduce on a Ubuntu 14.04 with a 4.4 kernel:

% docker info
Containers: 1
 Running: 0
 Paused: 0
 Stopped: 1
Images: 2
Server Version: 17.05.0-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins: 
 Volume: local
 Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 9048e5e50717ea4497b757314bad98ea3763c145
runc version: 9c2d8d184e5da67c95d601382adf14862e4f2228
init version: 949e6fa
Security Options:
 apparmor
Kernel Version: 4.4.0-83-generic
Operating System: Ubuntu 14.04.5 LTS
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 15.56GiB
Name: loki
ID: FQW3:U4O6:S5I7:JT35:YAPI:OE5P:FT2J:226W:DAVQ:DWUH:MIFE:ETFK
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

WARNING: No swap limit support
tonistiigi commented 7 years ago

@dmcgowan I don't think this is builder specific. If I move the image built in aufs to overlay it has the same problem. Starting container with docker run -v /var/lib/mysql also works fine(what should be recommended in a case like this).

geerlingguy commented 7 years ago

@tonistiigi - In a downstream issue where I was finding this same behavior on Docker for Mac, I added a /var/lib/mysql volume and the error went away. (See https://github.com/geerlingguy/drupal-vm/issues/1497).

That's an acceptable fix for me, and it's actually how I set up most projects' MySQL container in a Docker Compose... but until sometime in the past few weeks, I was able to not add that volume and still get MySQL to start up correctly.

aalexgabi commented 7 years ago

@tonistiigi I cannot use a volume because I use Docker for integration tests so I need the data snapshot inside the docker image so that the boot is fast and isolated from other containers. The alternative would be to manually manage MySQL volumes and copy the volume before each run but that would generate a lot of IO and lowering IO usage was the primary reason for using overlay2 in the first place.

The question for me is why this works with aufs and does not work with overlay2 and overlay? Is there any known limitation?

aalexgabi commented 7 years ago

I found the problem.

From the Docker documentation on overlay2:

open(2): OverlayFS only implements a subset of the POSIX standards. This can result in certain OverlayFS operations breaking POSIX standards. One such operation is the copy-up operation. Suppose that your application calls fd1=open("foo", O_RDONLY) and then fd2=open("foo", O_RDWR). In this case, your application expects fd1 and fd2 to refer to the same file. However, due to a copy-up operation that occurs after the second calling to open(2), the descriptors refer to different files. The fd1 continues to reference the file in the image (lowerdir) and the fd2 references the file in the container (upperdir). A workaround for this is to touch the files which causes the copy-up operation to happen. All subsequent open(2) operations regardless of read-only or read-write access mode will be referencing the file in the container (upperdir).

yum is known to be affected unless the yum-plugin-ovl package is installed. If the yum-plugin-ovl package is not available in your distribution such as RHEL/CentOS prior to 6.8 or 7.2, you may need to run touch /var/lib/rpm/* before running yum install. This package implements the touch workaround referenced above for yum.

As pointed in the docs the workaround is to touch the files used by MySQL:

FROM debian:wheezy

RUN apt-get update && apt-get install -y mysql-server-5.5
RUN find /var/lib/mysql -type f -exec touch {} \; && service mysql start

Can somebody add a note in the docs (next to the yum one) that mentions that MySQL is also affected?

AkihiroSuda commented 7 years ago

Anyone has straced this? Is this an open(2) issue or rename(2) one?

arku31 commented 6 years ago

For me it works only this way: cat init.sh #!/bin/bash find /var/lib/mysql -type f -exec touch {} \; && service mysql start tail Dockerfile -n 2 COPY init.sh /tmp/ ENTRYPOINT "/tmp/init.sh" && /bin/bash

Azeirah commented 6 years ago

The workaround works for me, but it takes quite a while to run on my Windows pc (~4-5 seconds). I'm using a mysql image that is specifically tailored to having super-fast start-up times, and without the touch workaround, it starts up in under a second, but with the fix, it takes over 5!

Is there anything else I can try as an alternative workaround on Windows?

sicheng20 commented 5 years ago

I have also encountered this when I excuted /etc/init.d/mysql start in docker:

190125 16:58:01 [ERROR] Can't open the mysql.plugin table. Please run mysql_upgrade to create it.
190125 16:58:01 InnoDB: The InnoDB memory heap is disabled
190125 16:58:01 InnoDB: Mutexes and rw_locks use GCC atomic builtins
190125 16:58:01 InnoDB: Compressed tables use zlib 1.2.8
190125 16:58:01 InnoDB: Using Linux native AIO
190125 16:58:01 InnoDB: Initializing buffer pool, size = 128.0M
190125 16:58:01 InnoDB: Completed initialization of buffer pool
190125 16:58:01 InnoDB: highest supported file format is Barracuda.
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
190125 16:58:01  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
190125 16:58:01  InnoDB: Waiting for the background threads to start
190125 16:58:02 InnoDB: 5.5.58 started; log sequence number 701019810
190125 16:58:02 [Note] Server hostname (bind-address): '127.0.0.1'; port: 3306
190125 16:58:02 [Note]   - '127.0.0.1' resolves to '127.0.0.1';
190125 16:58:02 [Note] Server socket created on IP: '127.0.0.1'.
190125 16:58:02 [ERROR] Fatal error: Can't open and lock privilege tables: Got error 140 from storage engine
190125 17:00:13 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead.
190125 17:00:13 [Note] Plugin 'FEDERATED' is disabled.
/usr/sbin/mysqld: Got error 140 from storage engine
190125 17:00:13 [ERROR] Can't open the mysql.plugin table. Please run mysql_upgrade to create it.
190125 17:00:13 InnoDB: The InnoDB memory heap is disabled
190125 17:00:13 InnoDB: Mutexes and rw_locks use GCC atomic builtins
190125 17:00:13 InnoDB: Compressed tables use zlib 1.2.8
190125 17:00:13 InnoDB: Using Linux native AIO
190125 17:00:13 InnoDB: Initializing buffer pool, size = 128.0M
190125 17:00:13 InnoDB: Completed initialization of buffer pool
190125 17:00:13 InnoDB: highest supported file format is Barracuda.
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
190125 17:00:13  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
190125 17:00:13  InnoDB: Waiting for the background threads to start
190125 17:00:14 InnoDB: 5.5.58 started; log sequence number 701019810
190125 17:00:14 [Note] Server hostname (bind-address): '127.0.0.1'; port: 3306
190125 17:00:14 [Note]   - '127.0.0.1' resolves to '127.0.0.1';
190125 17:00:14 [Note] Server socket created on IP: '127.0.0.1'.
190125 17:00:14 [ERROR] Fatal error: Can't open and lock privilege tables: Got error 140 from storage engine

My machine is centos7 and my docker storage engine is overlay2, this is output of docker info:

[root@slave119 ~]# docker info
Containers: 1
 Running: 0
 Paused: 0
 Stopped: 1
Images: 10
Server Version: 1.12.6
Storage Driver: overlay2
 Backing Filesystem: xfs
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: seccomp
Kernel Version: 3.10.0-514.26.2.el7.x86_64
Operating System: NewStart Carrier Grade Server Linux Core 1.3
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 23.37 GiB
Name: slave119
ID: P3UX:UX3G:HC7E:PW55:NI7L:Q53S:K57X:E4DK:WRXU:MGNO:CRV4:YQRA
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 16
 Goroutines: 24
 System Time: 2019-01-25T17:15:31.396644054+08:00
 EventsListeners: 0
Registry: https://index.docker.io/v1/
Insecure Registries:
 0.0.0.0/0
 127.0.0.0/8
MatrixManAtYrService commented 5 years ago

I found that this problem doesn't happen for me (Ubuntu 18.04) but it does happen for my colleagues that use docker for Mac OS. See the gist below for the output of docker info on these two systems:

https://gist.github.com/MatrixManAtYrService/77f433e2fa4d6e091d75a8394781be45