Shopify / ghostferry

The swiss army knife of live data migrations
https://shopify.github.io/ghostferry
MIT License
759 stars 72 forks source link

Extend inline verifier to do deep comparison of json objects #355

Closed mtaner closed 3 months ago

mtaner commented 3 months ago

Inline verifier checks were failing when binlog streamed event includes a json that contains a float value with trailing 0, e.g. {"amount": 15.0 }. go-mysql is parsing this with (json unmarshall/marshall) to {"amount": 15 }. As inline verifier is using checksum check for the whole row, this is failing, even though 15.0 & 15 can be considered the same. (this does not apply to values such as 15.123, these are all copied correctly.)

So as a fix if the checksum check fails, this logic will check for any json column and if they are present, then it will do a deep comparison between json values.

Very new to golang, so welcoming suggestions for improvements or anything I might have missed in the logic.