[ERROR] InnoDB: Upgrade after a crash is not supported.

[roncli]

Hello! I had a MariaDB v10.7.1 Docker container that I recently attempted to upgrade to 10.9.2. It failed with the following message on startup:

[ERROR] InnoDB: Upgrade after a crash is not supported. The redo log was created with MariaDB 10.7.1.

Downgrading the Docker container back to v10.7.1 allowed the database to run again, so I tried a smaller upgrade, first to v10.7.5. That works, and is what the container is running today.

Then I tried going to the next version available, 10.8.2, and that came back with the same error message:

[ERROR] InnoDB: Upgrade after a crash is not supported. The redo log was created with MariaDB 10.7.1.

Note it still says created with 10.7.1.

How can I get around this and upgrade to the latest version?

---

I should also note that I'm using MariaDB as the data storage for Photoprism, if that might be of any use.

[grooverdan]

You need to shutdown the 10.7.5 container cleanly first. What do the container shutdown logs look like for your standard shutdown mechanism?

[grooverdan]

For completeness what does the failed startup log look like as well.

[roncli]

What do the container shutdown logs look like for your standard shutdown mechanism?

There are none! And therein was the clue that got me to figure out what was going on! An explanation in case anyone else comes across this:

docker-compose down and docker stop send a SIGTERM, and then after 10 seconds, a SIGKILL. The commands were taking over 10 seconds, which meant the SIGKILL was happening. Why? Because I setup my container wrong.

In docker-compose.yml, I use this entrypoint:

entrypoint: /var/mariadb/start.sh

This allows me to add some command line switches I want. So far so good. However, here is start.sh, and where it starts to go wrong:

#!/bin/sh

#Run app.
/usr/local/bin/docker-entrypoint.sh mysqld --innodb-buffer-pool-size=256M --transaction-isolation=READ-COMMITTED --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci --max-connections=512 --innodb-rollback-on-timeout=OFF --innodb-lock-wait-timeout=120 --user=mysql

If you're not sure what's wrong with this, you can read https://madflojo.medium.com/shutdown-signals-with-docker-entry-point-scripts-5e560f4e2d45 for details. The TL;DR is that this makes PID 1 the script, and a higher PID is the server process. In order for SIGTERM to work correctly, PID 1 has to be the server process. To do that, you have to replace the script process with the server process using exec:

#!/bin/sh

#Run app.
exec /usr/local/bin/docker-entrypoint.sh mysqld --innodb-buffer-pool-size=256M --transaction-isolation=READ-COMMITTED --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci --max-connections=512 --innodb-rollback-on-timeout=OFF --innodb-lock-wait-timeout=120 --user=mysql

And that was it. Once I got a proper shutdown on 10.7.5 after using the script with exec, I was able to use the 10.9.2 image and it started up just fine, no errors.

Thanks Dan for nudging me in the right direction. I'm semi-new to Docker, and this was a good thing to learn!

[grooverdan]

A longer than 10 second shutdown isn't necessary caused by you. There are things mariadb does to avoid a crash recovery scenario that can take longer.

Note instead of doing a new entrypoint just putting the mariadbd command line options on a command: --innodb-buffer-pool-size=256M .... is sufficient with the default entrypoint. (--user=mysql is default so could be omitted).

Thanks for the reference for signals, that is good to know. Glad you worked it out too.