This PR fixes the source-mysql connector to support MySQL 8.4, which was released a couple of months ago. This involved a couple of trivial incompatibilities and a version bump to the CI database dockerfile, and then the main thing was that MySQL 8.4 went and fully removed the deprecated SHOW MASTER STATUS command.
This command was deprecated in MySQL 8.0.22 in favor of a new SHOW BINARY LOG STATUS which does the exact same thing. Since we want to support MySQL <8.0.22 we used the old query exclusively, but now that's no longer sufficient and there is no single query that works on all versions of MySQL. So to handle this we have to actually keep track of the database version and issue the appropriate query depending on what server we're connected to.
This was done by factoring out most of the version string querying/parsing logic from the prerequisiteVersion check into a helper function which gets run immediately upon connecting and which saves the resulting version info into some fields of the database struct, and then factoring out the SHOW MASTER STATUS query into another helper function which selects the appropriate query to run based on those properties.
This seems to work, although the refactor of the version-checking logic has me feeling a bit cautious so I'll probably do a careful rollout tomorrow where I'll force a restart of all production MySQL capture tasks and make sure none of them suddenly start failing, while I'm still sitting at my desk and thinking about it so it can be reverted quick if anything goes wrong.
(Honestly this keeps coming up, it might be time to write a helper script that can query the set of live tasks running a particular connector, figure out which ones are currently green, issue a forced restart to all of them, and then complain if any of them start failing during some short window after the restart. I know I'll want it for the Postgres "slot validation redux" PR whenever I get around to merging that...)
Description:
This PR fixes the
source-mysqlconnector to support MySQL 8.4, which was released a couple of months ago. This involved a couple of trivial incompatibilities and a version bump to the CI database dockerfile, and then the main thing was that MySQL 8.4 went and fully removed the deprecatedSHOW MASTER STATUScommand.This command was deprecated in MySQL 8.0.22 in favor of a new
SHOW BINARY LOG STATUSwhich does the exact same thing. Since we want to support MySQL <8.0.22 we used the old query exclusively, but now that's no longer sufficient and there is no single query that works on all versions of MySQL. So to handle this we have to actually keep track of the database version and issue the appropriate query depending on what server we're connected to.This was done by factoring out most of the version string querying/parsing logic from the
prerequisiteVersioncheck into a helper function which gets run immediately upon connecting and which saves the resulting version info into some fields of the database struct, and then factoring out theSHOW MASTER STATUSquery into another helper function which selects the appropriate query to run based on those properties.This seems to work, although the refactor of the version-checking logic has me feeling a bit cautious so I'll probably do a careful rollout tomorrow where I'll force a restart of all production MySQL capture tasks and make sure none of them suddenly start failing, while I'm still sitting at my desk and thinking about it so it can be reverted quick if anything goes wrong.
(Honestly this keeps coming up, it might be time to write a helper script that can query the set of live tasks running a particular connector, figure out which ones are currently green, issue a forced restart to all of them, and then complain if any of them start failing during some short window after the restart. I know I'll want it for the Postgres "slot validation redux" PR whenever I get around to merging that...)
This change is