python2 version of holland breaks on Mariadb 10.6

[soulen3]

Effects python2 version of holland including 1.1.22

Error Message:

Failed to connect to database
MySQL Error (2019, "Can't initialize character set utf8 (path: /usr/share/mysql/charsets/)")
Backup failed after 0.01 seconds

I'm assuming it's the following change in Mariadb https://mariadb.com/kb/en/mariadb-1061-release-notes/

Character Sets
The utf8 character set (and related collations) is now by default an alias for utf8mb3 rather than the other way around. It can be set to imply utf8mb4 by changing the value of the old_mode system variable (MDEV-8334)

Potential fix:

diff --git a/plugins/holland.lib.mysql/holland/lib/mysql/client/base.py b/plugins/holland.lib.mysql/holland/lib/mysql/client/base.py
index d1485a5..a57f4bb 100644
--- a/plugins/holland.lib.mysql/holland/lib/mysql/client/base.py
+++ b/plugins/holland.lib.mysql/holland/lib/mysql/client/base.py
@@ -538,4 +538,4 @@ def connect(config, client_class=AutoMySQLClient):
         except KeyError:
             LOG.warning("Skipping unknown parameter %s", key)
     # also, always use utf8
-    return client_class(charset="utf8", **args)
+    return client_class(**args)

[soulen3]

Using the mariabackup plugin seems to be a valid work around.

[ed-velez]

This still appears to be an issue when performing a backup via mysqldump w/ MariaDB.

[soulen3]

I don't really have plans to fix this. As I believe it only effects newer version of MariaDB, and python2 is EOL. Using the mariabackup plugin or upgrading to an OS that support python3 would be the preferred work arounds.

Is this effecting more than MariaDB 10.6+?

[petecassetta]

I think this is worth fixing. Although Python2 is EOL, it is still the default version on Red Hat/CentOS 7, which does not go EOL until June 2024 (over 2 years from now). MariaDB is the default MySQL flavor on that OS, and in general it has become very mainstream on all platforms. Because MariaDB's backward-compatibility is so good, we see many customers upgrading to very new versions. That will continue over the next 2+ years so this problem will get progressively bigger until we say goodbye to the last OS that uses Python2 by default. The fix you've proposed is simple and works reliably. It's kind of a pain to have to install Holland then manually remediate the code. And it's asking a lot of techs to remember to remediate Holland whenever they upgrade MariaDB to 10.6+.

[soulen3]

The simplicity of the change isn't the problem. We don't have a testing pipeline for python2 any more. This is a change to the MySQL client library, not to the mariadb plugin. So it will effect all database types. Without a testing pipeline I don't have the confidence to push this.

The python2 branch is there for any major breakage or security issues. I don't see this as either of those. If someone wants to build and maintain a new testing pipeline we can discuss that. But I don't think it's worth the effort.

Figuring out how to get holland to run on the supported version of python3 on EL7 seems like a better way to go. EPEL7 was missing a dependency the last time I looked.

[petecassetta]

I'm just coming from the pragmatic side. The vast majority of EL7 boxes I work on don't have Python 3 installed, and none of them use Python 3 as the default Python (that breaks yum, right?). When you just "yum install holland" on EL7 you get the Python 2 package.

But if your dev process doesn't allow for updates to the Python 2 branch, then I guess that's a no-go and we'll have to burden techs working on EL7 with a more complex install. I feel it will be simpler for techs to do a yum install and then just manually edit this one file than to do a git clone and make a working Python 3, so we'll just have to train our people to remediate this when MariaDB 10.6+ is being used.

[soulen3]

Installing python3 on EL7 devices is trivial and does not break yum. Not that it helps, because there's not holland3 package in EPEL7 yet.

I still think the simplest work around would be to use the mariabackup plugin. This doesn't require any code changes, and you would just need to install the additional plugin.

It's not that updating the python2 branch breaks the dev process. It's that there isn't a dev process for it. Feel free to build one, and I'll be happy to merge the code.

[petecassetta]

MariaBackup is an excellent program, but it's a very different kind of backup, requiring a very different kind of restore. For a database server running one huge database, it's great. For a db server running a lot of small databases, mysqldump is generally preferred since we can restore a single database when that's all that's needed.

Creating a build process to fix one line of code without anticipating any future development doesn't feel like a good use of time; I agree with you there. We'll just try to keep our techs informed that they have to manually fix that file after install when using holland-mysqldump with MariaDB 10.6. We should be monitoring all Holland backups, so this problem will be caught quickly after a Holland install or a MariaDB upgrade to 10.6+.