lathonez / clone-row

Python utility to clone a row in mysql, from a target to source database, field by field
http://lathonez.com/2016/clone-row/
MIT License
7 stars 1 forks source link

Encoding issue in backup files #1

Closed lathonez closed 9 years ago

lathonez commented 9 years ago
                "vegeNoFish": {\
                    "question": "Vegetarian <96> No Fish",\
                    "yesResponse": "Vegetarian <96> No Fish",\
                    "noResponse": ""\
                },\
                "vegeOkFish": {\
                    "question": "Vegetarian <96> Fish OK",\
                    "yesResponse": "Vegetarian <96> Fish OK",\
                    "noResponse": ""\
                },\

The data in the db looks like this:

                "vegeNoFish": {
                    "question": "Vegetarian – No Fish",
                    "yesResponse": "Vegetarian – No Fish",
                    "noResponse": ""
                },
                "vegeOkFish": {
                    "question": "Vegetarian – Fish OK",
                    "yesResponse": "Vegetarian – Fish OK",
                    "noResponse": ""
                },
lathonez commented 9 years ago

Simplest replication is this, outside of the script:

select * into outfile '/tmp/test.out' from client where pool = 'truebluesailingwhitsundays';
lathonez commented 9 years ago

Local:

+--------------------------+
| @@character_set_database |
+--------------------------+
| latin1                   |
+--------------------------+
1 row in set (0.00 sec)

Prod:

mysql>  SELECT @@character_set_database;
+--------------------------+
| @@character_set_database |
+--------------------------+
| utf8                     |
+--------------------------+
1 row in set (0.00 sec)
lathonez commented 9 years ago

We should probably print a warning (or fail altogether) if the source and target database encodings are different

lathonez commented 9 years ago

alter database '<databasename>' CHARACTER SET 'utf8';

lathonez commented 9 years ago

The above didn't do the trick, but this did:

alter table '<table_name>' convert to character set ut8 collate utf8_unicode_ci;

lathonez commented 9 years ago

So I think we should check the encoding of the two tables at the start of the app, and bail out if the encoding is different, basically we can't rely on the backups unless the encoding is the same so it isn't safe to carry on.

lathonez commented 9 years ago

https://github.com/lathonez/mysql-clone-row/commit/2fa1a20e719773a8af661c9f2370cdb3681d2d40

@pfirsowicz thanks a lot with your help on this one