Can't connect to mysql server because it starts with skip-networking=TRUE

[jones1008]

I have the following docker-compose.yml setup:

services:
  db:
    image: jbergstroem/mariadb-alpine:10.4.15
    environment:
      MYSQL_ROOT_PASSWORD: root
      MYSQL_DATABASE: test
    volumes:
      - ./db:/docker-entrypoint-initdb.d
    ports:
      - "3306:3306"

At intial startup it loads the file ./db/test.sql into the database test. After the container starts, I try to connect to it with a local mysql client installation:

>mysql -uroot -proot
ERROR 2013 (HY000): Lost connection to MySQL server at 'reading initial communication packet', system error: 2 "No such file or directory"

and on the Docker output I get:

Version: '10.4.15-MariaDB'  socket: '/run/mysqld/mysqld.sock'  port: 0  MariaDB Server

Note the port is set to 0 instead of 3306. A quick search states that --skip-networking might be enabled. And in fact the sh/run.sh script starts the temporary mysql daemon with the --skip-networking flag to import my test.sql.

My guess is that mysqld somehow saves the variable skip-networking internally and keeps it to TRUE when executing mysqld in the last line of run.sh

A solution might be to explicitly set --skip-networking=FALSE in the last line of run.sh. Might be filing a PR soon.

Also note that somehow this is not an issue in the latest mariadb version 10.4.17... So changing the docker-compose.yml to the following fixes the issue:

services:
  db:
    image: jbergstroem/mariadb-alpine:10.4.17
    # ...

[jbergstroem]

Thanks for filing a bug report! Will investigate and try to reproduce.

[jones1008]

added a fix in PR #57

[jbergstroem]

I haven't been able to reproduce this. Is it happening 100% of the time for you?

[jones1008]

I just reproduced this on another Windows machine... It still starts with port: 0 when the container imports the ./db/test.sql, so I guess it is happening 100% of the time:

init: installing mysql client
init: updating system tables
init: adding /docker-entrypoint-initdb.d/test.sql
init: removing mysql client
[...]
Version: '10.4.15-MariaDB'  socket: '/run/mysqld/mysqld.sock'  port: 0  MariaDB Server

Please note, that I am using the image version 10.4.15 in my docker-compose.yml and not the latest version 10.4.17. Also: It didn't matter if I used the WSL2 based docker engine on Windows or the Hyper-V version, same problem for both...

[jbergstroem]

Ok, thanks for verifying. Since I aim to maintain latest branches of 10.4.x, 10.5.x (and possibly 10.3, 10.2); I'm unsure if I should merge a patch for a bug that's fixed upstream. What do you think?

[jones1008]

Well, if you want to support multiple branches of mariadb I think you want to fix this... And since the default value of --skip-networking is OFF anyway and I am only explicitly calling mysqld with this flag set to false in my PR, I don't see a reason not to merge this.

[Dashman99]

I see this same issue on my Windows machine using WSL2, with both image versions 10.1.16 and 10.5. What is the workaround?

Version: '10.5.15-MariaDB-1:10.5.15+maria~focal' socket: '/run/mysqld/mysqld.sock' port: 0 mariadb.org binary distribution

compose file:

db:
    image: mariadb:10.5
    restart: always
    env_file:
      - ./db.env
    volumes: 
      - ./db/init:/docker-entrypoint-initdb.d
    ports:
      - 3309:3306`

I also noticed that when I run bash in my db container and get a process list, I see the '--skip-networking' flag

root@8327183c80f5:/# ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
mysql        1  0.0  0.0   4352  3516 ?        Ss   14:38   0:00 /bin/bash /usr/local/bin/docker-entrypoint.sh mysqld
mysql      104 79.4  6.0 1861904 242776 ?      Sl   14:38   2:38 mysqld --skip-networking --default-time-zone=SYSTEM --socket=/run/mysqld/mysqld.sock --wsrep_on=OFF --loose-innodb_buffer_pool_load_at_startup=0

[Dashman99]

I noticed that my docker-compose file and @jones1008 both were using the docker-entrypoint-initdb.d option so I just tested with commenting this out and mysqld runs on port 3306.

docker-compose output

db_1       | 2022-05-09 14:55:37 0 [Note] mysqld: ready for connections.
db_1       | Version: '10.5.15-MariaDB-1:10.5.15+maria~focal'  socket: '/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution

docker-compose file

 db:
    image: mariadb:10.5
    restart: always
    env_file:
      - ./db.env
#    volumes: 
#      - ./db/init:/docker-entrypoint-initdb.d
    ports:
      - 3309:3306

Here is the process list from the db container

root@c8df207a315a:/# ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
mysql        1  0.0  2.0 1345568 84476 ?       Ssl  14:55   0:00 mysqld
root       157  1.5  0.0   4244  3516 pts/0    Ss   15:03   0:00 bash
root       165  0.0  0.0   5896  2872 pts/0    R+   15:03   0:00 ps aux

[jbergstroem]

Sorry for lagging behind here; its hard for me to reproduce it since I don't have any windows environments. I will see if I can get a VM going this weekend to debug this further.

Was hoping to get tests going in github actioins but are at a loss on how to set up a full environment. Any input much appreciated!

[SmartFinn]

I faced with the same issue on podman (3.0.1 on Debian 11, and 4.2.1 on Fedora 36):

podman run --rm --env-file=init.env --pod=mariadb-pod --volume=$HOME/Development/camilla/dockerfiles/mariadb-alpine/init:/docker-entrypoint-initdb.d --volume=mariadb-data:/var/lib/mysql --name mariadb docker.io/jbergstroem/mariadb-alpine

Output

init: installing mysql client init: updating system tables init: ignoring /docker-entrypoint-initdb.d/_backup_db: not a recognized format init: executing /docker-entrypoint-initdb.d/giteadb.sh init: executing /docker-entrypoint-initdb.d/pigallery2.sh init: executing /docker-entrypoint-initdb.d/releasebell.sh init: removing mysql client 2022-10-12 6:51:08 0 [Note] /usr/bin/mariadbd (server 10.6.8-MariaDB) starting as process 1 ... 2022-10-12 6:51:08 0 [Note] InnoDB: Compressed tables use zlib 1.2.12 2022-10-12 6:51:08 0 [Note] InnoDB: Number of pools: 1 2022-10-12 6:51:08 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions 2022-10-12 6:51:08 0 [Note] mariadbd: O_TMPFILE is not supported on /var/tmp (disabling future attempts) 2022-10-12 6:51:08 0 [Note] InnoDB: Using Linux native AIO 2022-10-12 6:51:08 0 [Note] InnoDB: Initializing buffer pool, total size = 10485760, chunk size = 10485760 2022-10-12 6:51:08 0 [Note] InnoDB: Completed initialization of buffer pool 2022-10-12 6:51:08 0 [Note] InnoDB: 128 rollback segments are active. 2022-10-12 6:51:08 0 [Note] InnoDB: Creating shared tablespace for temporary tables 2022-10-12 6:51:08 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... 2022-10-12 6:51:08 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB. 2022-10-12 6:51:08 0 [Note] InnoDB: 10.6.8 started; log sequence number 42185; transaction id 14 2022-10-12 6:51:08 0 [Note] Plugin 'FEEDBACK' is disabled. 2022-10-12 6:51:08 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool 2022-10-12 6:51:08 0 [Note] InnoDB: Buffer pool(s) load completed at 221012 6:51:08 2022-10-12 6:51:08 0 [Note] /usr/bin/mariadbd: ready for connections. Version: '10.6.8-MariaDB' socket: '/run/mysqld/mysqld.sock' port: 0 MariaDB Server

Without /docker-entrypoint-initdb.d mounting, it works correctly.

[jbergstroem]

Without /docker-entrypoint-initdb.d mounting, it works correctly.

That's very interesting. Will try to reproduce.

[jbergstroem]

I'm just gonna remove --skip-networking. It is an unnecessary optimization. I'm not able to reproduce this or confirm that the temporary mysqld spun up is polluting a config somehow, but its clearly unnecessary.

[jbergstroem]

Ok, pushed the change. Let me know if that works for you (should you be in a position to test main - otherwise wait for next release!).

[darkhacknet]

Guys can you please help me ? Seems to be impossible to set this as 0 or false ! skip-networking=TRUE

please help me to understan how to do that ?

any help will be appreaciate it. thanks in advance !

[jbergstroem]

@darkhacknet I removed the flag from the container. Are you on 10.6.10? What issues are you facing? Could you open a new issue so we can track it separately?