Cannot reset Root Password

[shumbashi]

I have a 5 node Galera cluster running on swarm using this repo. I wanted to reset the root password. The documentation mentions that "Galera container should be stopped" so I have scaled down the mysql_node service to 0 (making sure all Galera containers are shut down) and then attempted to run the following container from one of the swarm nodes

docker run --rm -v /dbdata:/var/lib/mysql --name no-galera-temp colinmollenhour/mariadb-galera-swarm:10.2 no-galera --skip-grant-tables

The container quickly exits with the following error

Starting with Galera disabled
2020-06-11  8:32:52 139777314232256 [Note] mysqld (mysqld 10.2.18-MariaDB-1:10.2.18+maria~bionic) starting as process 7 ...
2020-06-11  8:32:52 139777314232256 [Warning] You need to use --log-bin to make --binlog-format work.
2020-06-11  8:32:52 139777314232256 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2020-06-11  8:32:52 139777314232256 [Note] InnoDB: Uses event mutexes
2020-06-11  8:32:52 139777314232256 [Note] InnoDB: Compressed tables use zlib 1.2.11
2020-06-11  8:32:52 139777314232256 [Note] InnoDB: Using Linux native AIO
2020-06-11  8:32:52 139777314232256 [Note] InnoDB: Number of pools: 1
2020-06-11  8:32:52 139777314232256 [Note] InnoDB: Using SSE2 crc32 instructions
2020-06-11  8:32:52 139777314232256 [Note] InnoDB: Initializing buffer pool, total size = 256M, instances = 1, chunk size = 128M
2020-06-11  8:32:52 139777314232256 [Note] InnoDB: Completed initialization of buffer pool
2020-06-11  8:32:52 139776573703936 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2020-06-11  8:32:52 139777314232256 [Note] InnoDB: Highest supported file format is Barracuda.
2020-06-11  8:32:53 139777314232256 [Note] InnoDB: 128 out of 128 rollback segments are active.
2020-06-11  8:32:53 139777314232256 [Note] InnoDB: Creating shared tablespace for temporary tables
2020-06-11  8:32:53 139777314232256 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2020-06-11  8:32:53 139777314232256 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2020-06-11  8:32:53 139777314232256 [Note] InnoDB: 5.7.23 started; log sequence number 35431847949
2020-06-11  8:32:53 139776195806976 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
2020-06-11  8:32:53 139776195806976 [Note] InnoDB: Buffer pool(s) load completed at 200611  8:32:53
2020-06-11  8:32:53 139777314232256 [Note] Plugin 'FEEDBACK' is disabled.
2020-06-11  8:32:53 139777314232256 [Note] Recovering after a crash using tc.log
2020-06-11  8:32:53 139777314232256 [ERROR] Recovery failed! You must enable all engines that were enabled at the moment of the crash
2020-06-11  8:32:53 139777314232256 [ERROR] Crash recovery failed. Either correct the problem (if it's, for example, out of memory error) and restart, or delete tc log and start mysqld with --tc-heuristic-recover={commit|rollback}
2020-06-11  8:32:53 139777314232256 [ERROR] Can't init tc log
2020-06-11  8:32:53 139777314232256 [ERROR] Aborting

Can you please advise if I'm doing this wrong or what?

[colinmollenhour]

I googled "Can't init tc log" and found this. I suggest trying to add --log-bin to the command options.

[oliverlj]

same error when I tried to update from 10.2 to 10.4 How to do a container upgrade from 10.2 to 10.4 ?

[oliverlj]

I got Received TERM|INT signal. in no-galera mode, but i didn't sent anythinh. maybe a heathcheck not working ?

[oliverlj]

I used

no-galera --log-bin=mysqld-bin

Then stared the cluster normally and now upgrades has been done

[colinmollenhour]

In general if you are extending the pre-built container image with your own config you will want to use your custom image also for the no-galera command so that the server starts up with the exact same configuration. Is that the case that your regular nodes run an image with log_bin specified already in the config?

[oliverlj]

I made a deployment with log-bin param

node:
    image: colinmollenhour/mariadb-galera-swarm:10.4.11-2020-01-06
    environment:
      - HEALTHY_WHILE_BOOTING=1
      - XTRABACKUP_PASSWORD_FILE=/run/secrets/xtrabackup_password
    networks:
      - back
    command: >
      no-galera
      --log-bin=mysqld-bin
    volumes:
      - {{ ha_dir }}/{% raw %}{{.Service.Name}}-{{.Task.Slot}}-data{% endraw %}:/var/lib/mysql
    deploy:
      replicas: 1
    secrets:
      - xtrabackup_password

[colinmollenhour]

Strange, so MariaDb will start with Galera without log-bin but will not start without Galera without log-bin?

I didn't add log_bin to the default config because I figured people will want to customize it (e.g. special purpose volumes) but I would assume that everyone will run it in production.

[colinmollenhour]

Updated README.md with mention of log-bin parameter.