search

codership / mysql-wsrep

wsrep API patch for MySQL server
Other
65 stars 34 forks source link

Node consistency compromised during shutdown #353

Closed mkaruza closed 5 years ago

mkaruza commented 5 years ago

MySQL 5.7 shutdown procedure does not wait till all applier threads are closed. If applier is applying write set during initiated shutdown it could happen that node consistency is compromised because write set is not applied resulting in node abort.

... 2019-05-29T05:00:35.889463Z 0 [Note] Shutting down slave threads 2019-05-29T05:00:36.145648Z 2 [Warning] WSREP: Failed to apply app buffer: seqno: 547980330, status: 1 at galera/src/trx_handle.cpp:apply():353 Retrying 2th time 2019-05-29T05:00:36.145671Z 2 [Note] WSREP: applier has been aborted, skipping apply_rbr: 547980330 2019-05-29T05:00:36.176279Z 2 [Warning] WSREP: Failed to apply app buffer: seqno: 547980330, status: 1 at galera/src/trx_handle.cpp:apply():353 Retrying 3th time 2019-05-29T05:00:36.176302Z 2 [Note] WSREP: applier has been aborted, skipping apply_rbr: 547980330 2019-05-29T05:00:36.251995Z 2 [Warning] WSREP: Failed to apply app buffer: seqno: 547980330, status: 1 at galera/src/trx_handle.cpp:apply():353 Retrying 4th time 2019-05-29T05:00:36.252027Z 2 [Note] WSREP: applier has been aborted, skipping apply_rbr: 547980330 2019-05-29T05:00:36.369721Z 2 [ERROR] WSREP: Failed to apply trx: source: 8ec31469-8109-11e9-9595-3a236f1a3e8c version: 4 local: 0 state: APPLYING flags: 1 conn_id: 1951034 trx_id: 140947703 seqnos (l: 41027348, g: 547980330, s: 547980135, d: 547980098, ts: 10797851157436590) 2019-05-29T05:00:36.369780Z 2 [ERROR] WSREP: Failed to apply trx 547980330 4 times 2019-05-29T05:00:36.369795Z 2 [ERROR] WSREP: Node consistency compromised, aborting... 2019-05-29T05:00:36.369814Z 2 [Note] WSREP: /usr/sbin/mysqld: Terminated. ...

mkaruza commented 5 years ago

This should be fixed in upcoming 5.7.26-25.18