github / gh-ost

GitHub's Online Schema-migration Tool for MySQL
MIT License
12.39k stars 1.26k forks source link

Test gh-ost on master, the parameters have been added parameters output invalid #493

Open boomballa opened 7 years ago

boomballa commented 7 years ago

Example 1:

[shell ~]# gh-ost -user="root" -password="pwd4mysql" -host="172.16.3.88" -port=3306 -database="crm"  -table="dish" -alter="drop colomn archer_test_column"   –allow-on-master -switch-to-rbr  –cut-over=default –default-retries=120 –throttle-control-replicas="172.16.3.95"  –max-lag-millis=1500  –verbose -debug  -execute 
2017-09-07 17:32:20 FATAL You must be using ROW binlog format. I can switch it for you, provided --switch-to-rbr and that 172.16.3.88:3306 doesn't have replicas

Just in the example above, I have added the -switch-to-rbr parameter, and I also have from the replicas, do not know gh-ost is what principle to use to check from the replicas?

mysql> show slave hosts;
+-----------+------+------+-----------+--------------------------------------+
| Server_id | Host | Port | Master_id | Slave_UUID                           |
+-----------+------+------+-----------+--------------------------------------+
|    943308 |      | 3306 |    999999 | 2fa13881-5a1e-11e7-a3d5-000c2902e7e6 |
|  99993306 |      | 3306 |    999999 | 64dbbdec-5a1f-11e7-a3dd-000c29cb5c36 |
+-----------+------+------+-----------+--------------------------------------+
2 rows in set (0.00 sec)

Example 2:

[shell ~]# gh-ost –allow-on-master –max-load=Threads_running=25 –critical-load=Threads_running=1000 –chunk-size=1000 –throttle-control-replicas="172.16.3.95:3306" –max-lag-millis=1500 -user="root" -password="pwd4mysql" -host="172.16.3.88"   -database="crm"  -table="dish" –verbose -alter="drop colomn archer_test_column"    –cut-over=default –default-retries=120 –switch-to-rbr –panic-flag-file=/tmp/ghost.panic.flag –postpone-cut-over-flag-file=/tmp/ghost.postpone.flag  –execute
2017-09-07 17:41:31 FATAL --database must be provided and database name must not be empty

I have added the -database parameters, why suggested that my database parameters empty? The first test gh-ost, do not know if the method is not used? I hope everyone and the author to help.

tomkrouper commented 7 years ago

Example 1: ... snip ... 2017-09-07 17:32:20 FATAL You must be using ROW binlog format. I can switch it for you, provided --switch-to-rbr and that 172.16.3.88:3306 doesn't have replicas

Can you run SHOW GLOBAL VARIABLES LIKE 'binlog_format'; on 172.16.3.88? From your running of SHOW SLAVE HOSTS (I'm assuming on 172.16.3.88 you've already proven the second part of the error message false.) I think the issue is gh-ost won't change a host to RBR if it has replicas. So your test on master won't work. I think you need to change to RBR on the master manually. (After verifying and/or changing it on the replicas.)

tomkrouper commented 7 years ago

Example 2: ... snip ... 2017-09-07 17:41:31 FATAL --database must be provided and database name must not be empty

Try --database instead of -database. I know there is another discussion about whether we use a single dash or double. (We are going to try to get it consistent.)

boomballa commented 7 years ago

Thank you for your answers and suggestions first.

Example 1:

the binlog_format on the mainframe is not Row format on the host which ip is 172.16.3.88, I think in the master on the use of -allow-on-master and -switch-to-rbr parameters can be changed. This is my misuse of the use of parameters. So i can understand into the binlog is not Row format master can not use gh-ost tools? This is also used in the use of certain limitations.

Example 2:

[shell ~]# gh-ost –allow-on-master –max-load=Threads_running=25 –critical-load=Threads_running=1000 –chunk-size=1000 –throttle-control-replicas="172.16.3.95:3306" –max-lag-millis=1500 -user="root" -password="pwd4mysql" -host="172.16.3.88"   --database="crm"  -table="dish" –verbose -alter="drop colomn archer_test_column"    –cut-over=default –default-retries=120 –switch-to-rbr –panic-flag-file=/tmp/ghost.panic.flag –postpone-cut-over-flag-file=/tmp/ghost.postpone.flag  –execute
2017-09-08 01:52:05 FATAL --database must be provided and database name must not be empty

I try --database instead of -database ,But still can not work properly,and the same error output.

Yes ah, use a single dash or double is also my more confusing question, it is not good to determine which one to use.

tomkrouper commented 7 years ago

Example 1:

If --host is not RBR and has replicas on it, --switch-to-rbr will not work.

Example 2:

Try using -- on all the options. If it works after that, then we definitely have a bug we need to fix. Though I think we are already going to standardize based on the discussion on https://github.com/github/gh-ost/issues/127.

shlomi-noach commented 7 years ago

The golang parser (flag package) supports both notations, and unfortunately golang decided that single hyphens are the standard (mostly contradicting industry standard).

-database and --database should act the same. If not, then truly something is terribly wrong.

Re: RBR, have you seen this doc? https://github.com/github/gh-ost/blob/master/doc/migrating-with-sbr.md

shlomi-noach commented 7 years ago

with regard FATAL --database must be provided and database name must not be empty, one thing I notice is that there's extra spaces before and after the --database="crm" flag, and I wonder if there aren't any invisible control characters which break the parsing of the flags. If you're able to rewrite the entire command in a simple text editor without copy+paste the --database="crm" to confirm that would be helpful.

boomballa commented 7 years ago

Thank you for your answer, I re-enter the order manually, it works fine. However, I am more helpless problem is that my side of the production environment database, binlog format are all mixed, whether it is doomed I can not use this tool?

017-09-13 10:55:12 INFO starting gh-ost 1.0.40
2017-09-13 10:55:12 INFO Migrating `crm`.`dish`
2017-09-13 10:55:12 INFO connection validated on 172.16.3.88:3306
2017-09-13 10:55:12 INFO User has ALL privileges
2017-09-13 10:55:12 FATAL 172.16.3.88:3306 has MIXED binlog_format, but I'm too scared to change it to ROW because it has replicas. Bailing out
shlomi-noach commented 7 years ago

@boomballa you are not doomed; when we developed gh-ost we were running STATEMENT based replication. Again, have you read this doc? https://github.com/github/gh-ost/blob/master/doc/migrating-with-sbr.md All you need is a single replica to use with ROW.

boomballa commented 7 years ago

Late to apologize, I carefully read the document, before I see not careful, neglected. I also carefully look at other safe, thanks about everyone's answer.