sysown / proxysql

High-performance MySQL proxy with a GPL license.
http://www.proxysql.com
GNU General Public License v3.0
5.96k stars 972 forks source link

when mysql-multiplexing='true' some transactions cannot rollback #2728

Open nasamitnick opened 4 years ago

nasamitnick commented 4 years ago

proxysql version:2.0.10-27-g5b31997 os version:redhat 7.5

set autocommit variables :

+---------------------------------------+----------------+
| variable_name                         | variable_value |
+---------------------------------------+----------------+
| mysql-forward_autocommit              | true           |
| mysql-enforce_autocommit_on_reads     | false          |
| mysql-autocommit_false_not_reusable   | true           |
| mysql-autocommit_false_is_transaction | true           |
+---------------------------------------+----------------+

runtime_mysql_users:

*************************** 1. row ***************************
              username: uip_sit
              password: *AFFB3BE27A5EF1B53F42BCC3AB72F81786B22B9A
                active: 1
               use_ssl: 0
     default_hostgroup: 10
        default_schema: 
         schema_locked: 0
transaction_persistent: 1
          fast_forward: 0
               backend: 1
              frontend: 0
       max_connections: 1000
               comment: 
*************************** 2. row ***************************
              username: uip_sit
              password: *AFFB3BE27A5EF1B53F42BCC3AB72F81786B22B9A
                active: 1
               use_ssl: 0
     default_hostgroup: 10
        default_schema: 
         schema_locked: 0
transaction_persistent: 1
          fast_forward: 0
               backend: 0
              frontend: 1
       max_connections: 1000
               comment: 
2 rows in set (0.00 sec)
#clear stats_mysql_query_digest table;
select * from stats_mysql_query_digest_reset\G
#Execute the following SQL 100 times
set autocommit = 0;
insert into admin_sm_team_member (TEAM_ID, USER_CODE, REMARK, CREATE_USER, CREATE_DT, STATUS) values (?, ?, ?, ?, ?, ?);
rollback;
#select  stats_mysql_query_digest
admin@127.0.0.1 : (none) 14:29:29> select * from stats_mysql_query_digest\G
*************************** 1. row ***************************
        hostgroup: 10
       schemaname: sbtest
         username: sbtest
   client_address: 
           digest: 0xDD3B740D393F9FA4
      digest_text: rollback
       count_star: 100
       first_seen: 1588573783
        last_seen: 1588573783
         sum_time: 46362
         min_time: 110
         max_time: 2678
sum_rows_affected: 0
    sum_rows_sent: 0
*************************** 2. row ***************************
        hostgroup: 30
       schemaname: sbtest
         username: sbtest
   client_address: 
           digest: 0xD6639786E36A2800
      digest_text: select @@session.transaction_read_only
       count_star: 200
       first_seen: 1588573783
        last_seen: 1588573783
         sum_time: 56641
         min_time: 126
         max_time: 1137
sum_rows_affected: 0
    sum_rows_sent: 200
*************************** 3. row ***************************
        hostgroup: 10
       schemaname: sbtest
         username: sbtest
   client_address: 
           digest: 0x375AF1F884A370FB
      digest_text: insert into admin_sm_team_member (TEAM_ID, USER_CODE, REMARK, CREATE_USER, CREATE_DT, STATUS) values (?, ?, ?, ?, ?, ?)
       count_star: 200
       first_seen: 1588573783
        last_seen: 1588573783
         sum_time: 81809
         min_time: 176
         max_time: 2615
sum_rows_affected: 100
    sum_rows_sent: 100
*************************** 4. row ***************************
        hostgroup: 10
       schemaname: sbtest
         username: sbtest
   client_address: 
           digest: 0xC51BDBBD6C9BB1D7
      digest_text: SET autocommit=?
       count_star: 200
       first_seen: 1588573783
        last_seen: 1588573783
         sum_time: 110319
         min_time: 341
         max_time: 1417
sum_rows_affected: 0
    sum_rows_sent: 0
4 rows in set (0.00 sec)

in stats_mysql_query_digest table We can see 100 times rollback Executed,but if set mysql-multiplexing='true' then some insert will be commit; if set mysql-multiplexing='false' then all insert will be rollback

I checked the document for Multiplexing,in document:

active transaction
When a transaction is active in a connection, multiplexing is disabled until the transaction commits or rollbacks.

Is this a bug? @renecannao

renecannao commented 4 years ago

Can you try the same using the default autocommit settings?

nasamitnick commented 4 years ago

I can't,because the Java framework spring transaction use set autocommit =0 start transaction.

So I used the following settings set autocommit variables :

+---------------------------------------+----------------+
| variable_name                         | variable_value |
+---------------------------------------+----------------+
| mysql-forward_autocommit              | true           |
| mysql-enforce_autocommit_on_reads     | false          |
| mysql-autocommit_false_not_reusable   | true           |
| mysql-autocommit_false_is_transaction | true           |
+---------------------------------------+----------------+

Is there a problem with mysql-autocommit_false_is_transaction=true @renecannao

renecannao commented 4 years ago

You are probably configuring the wrong variables. With default settings, autocommit=0 starts a transaction at the first non SELECT. If you want to start a transaction on the first SELECT, configure mysql-enforce_autocommit_on_reads=true. The other settings are unrelated to Java Spring

nasamitnick commented 4 years ago

thanks,i try

nasamitnick commented 4 years ago

use default settings like this set autocommit variables :

+---------------------------------------+----------------+
| variable_name                         | variable_value |
+---------------------------------------+----------------+
| mysql-forward_autocommit              | false           |
| mysql-enforce_autocommit_on_reads     | false          |
| mysql-autocommit_false_not_reusable   | false           |
| mysql-autocommit_false_is_transaction | false           |
+---------------------------------------+----------------+

Exce sql

  1. select
  2. update the update sql will route to reader_hostgroup,cause read only error,About this question see #2592 @renecannao
renecannao commented 4 years ago

Please attach the whole configuration, the whole error log, and a reproducible test case starting from a restarted proxysql. The reason why DML are started on reader hostgroup is likely because you are not using the default autocommit settings.

nasamitnick commented 4 years ago

i use default autocommit settings test,when spring datasource set prepstmts= true,Read only error occurred

1.select 2.update the update sql will route to reader_hostgroup,cause read only error,About this question see #2592 @renecannao

set prepstmts=false It's right route to write_hostgroup;

+--------------------------------------------------------------+-----------------------------+
| variable_name                                                | variable_value              |
+--------------------------------------------------------------+-----------------------------+
| admin-admin_credentials                                      | admin:admin                 |
| admin-checksum_mysql_query_rules                             | true                        |
| admin-checksum_mysql_servers                                 | true                        |
| admin-checksum_mysql_users                                   | true                        |
| admin-cluster_check_interval_ms                              | 1000                        |
| admin-cluster_check_status_frequency                         | 10                          |
| admin-cluster_mysql_query_rules_diffs_before_sync            | 3                           |
| admin-cluster_mysql_query_rules_save_to_disk                 | true                        |
| admin-cluster_mysql_servers_diffs_before_sync                | 3                           |
| admin-cluster_mysql_servers_save_to_disk                     | true                        |
| admin-cluster_mysql_users_diffs_before_sync                  | 3                           |
| admin-cluster_mysql_users_save_to_disk                       | true                        |
| admin-cluster_password                                       |                             |
| admin-cluster_proxysql_servers_diffs_before_sync             | 3                           |
| admin-cluster_proxysql_servers_save_to_disk                  | true                        |
| admin-cluster_username                                       |                             |
| admin-hash_passwords                                         | true                        |
| admin-mysql_ifaces                                           | 0.0.0.0:6032                |
| admin-read_only                                              | false                       |
| admin-refresh_interval                                       | 2000                        |
| admin-restapi_enabled                                        | false                       |
| admin-restapi_port                                           | 6070                        |
| admin-stats_credentials                                      | stats:stats                 |
| admin-stats_mysql_connection_pool                            | 60                          |
| admin-stats_mysql_connections                                | 60                          |
| admin-stats_mysql_query_cache                                | 60                          |
| admin-stats_mysql_query_digest_to_disk                       | 0                           |
| admin-stats_system_cpu                                       | 60                          |
| admin-stats_system_memory                                    | 60                          |
| admin-telnet_admin_ifaces                                    | (null)                      |
| admin-telnet_stats_ifaces                                    | (null)                      |
| admin-vacuum_stats                                           | true                        |
| admin-version                                                | 2.0.10-27-g5b31997          |
| admin-web_enabled                                            | false                       |
| admin-web_port                                               | 6080                        |
| mysql-add_ldap_user_comment                                  |                             |
| mysql-auditlog_filename                                      |                             |
| mysql-auditlog_filesize                                      | 104857600                   |
| mysql-aurora_max_lag_ms_only_read_from_replicas              | 2                           |
| mysql-auto_increment_delay_multiplex                         | 5                           |
| mysql-autocommit_false_is_transaction                        | false                       |
| mysql-autocommit_false_not_reusable                          | false                       |
| mysql-automatic_detect_sqli                                  | 0                           |
| mysql-binlog_reader_connect_retry_msec                       | 3000                        |
| mysql-client_found_rows                                      | true                        |
| mysql-client_multi_statements                                | true                        |
| mysql-client_session_track_gtid                              | true                        |
| mysql-commands_stats                                         | true                        |
| mysql-connect_retries_delay                                  | 1                           |
| mysql-connect_retries_on_failure                             | 10                          |
| mysql-connect_timeout_server                                 | 3000                        |
| mysql-connect_timeout_server_max                             | 10000                       |
| mysql-connection_delay_multiplex_ms                          | 0                           |
| mysql-connection_max_age_ms                                  | 0                           |
| mysql-connpoll_reset_queue_length                            | 50                          |
| mysql-default_character_set_results                          | utf8mb4                     |
| mysql-default_charset                                        | utf8mb4                     |
| mysql-default_collation_connection                           | utf8mb4_general_ci          |
| mysql-default_isolation_level                                | READ COMMITTED              |
| mysql-default_max_join_size                                  | 18446744073709551615        |
| mysql-default_max_latency_ms                                 | 1000                        |
| mysql-default_net_write_timeout                              | 60                          |
| mysql-default_query_delay                                    | 0                           |
| mysql-default_query_timeout                                  | 36000000                    |
| mysql-default_reconnect                                      | true                        |
| mysql-default_schema                                         | information_schema          |
| mysql-default_session_track_gtids                            | OFF                         |
| mysql-default_sql_auto_is_null                               | OFF                         |
| mysql-default_sql_safe_updates                               | OFF                         |
| mysql-default_sql_select_limit                               | DEFAULT                     |
| mysql-default_time_zone                                      | SYSTEM                      |
| mysql-default_transaction_read                               | WRITE                       |
| mysql-default_tx_isolation                                   | READ-COMMITTED              |
| mysql-enforce_autocommit_on_reads                            | false                       |
| mysql-eventslog_default_log                                  | 0                           |
| mysql-eventslog_filename                                     |                             |
| mysql-eventslog_filesize                                     | 104857600                   |
| mysql-eventslog_format                                       | 1                           |
| mysql-firewall_whitelist_enabled                             | 0                           |
| mysql-firewall_whitelist_errormsg                            | Firewall blocked this query |
| mysql-forward_autocommit                                     | false                       |
| mysql-free_connections_pct                                   | 10                          |
| mysql-handle_unknown_charset                                 | 1                           |
| mysql-have_compress                                          | true                        |
| mysql-have_ssl                                               | false                       |
| mysql-hostgroup_manager_verbose                              | 1                           |
| mysql-init_connect                                           |                             |
| mysql-interfaces                                             | 0.0.0.0:6033                |
| mysql-keep_multiplexing_variables                            | tx_isolation,version        |
| mysql-kill_backend_connection_when_disconnect                | true                        |
| mysql-ldap_user_variable                                     |                             |
| mysql-log_unhealthy_connections                              | true                        |
| mysql-long_query_time                                        | 1000                        |
| mysql-max_allowed_packet                                     | 104857600                   |
| mysql-max_connections                                        | 4000                        |
| mysql-max_stmts_cache                                        | 10000                       |
| mysql-max_stmts_per_connection                               | 20                          |
| mysql-max_transaction_time                                   | 14400000                    |
| mysql-min_num_servers_lantency_awareness                     | 1000                        |
| mysql-mirror_max_concurrency                                 | 16                          |
| mysql-mirror_max_queue_length                                | 32000                       |
| mysql-monitor_connect_interval                               | 60000                       |
| mysql-monitor_connect_timeout                                | 600                         |
| mysql-monitor_enabled                                        | true                        |
| mysql-monitor_galera_healthcheck_interval                    | 5000                        |
| mysql-monitor_galera_healthcheck_max_timeout_count           | 3                           |
| mysql-monitor_galera_healthcheck_timeout                     | 800                         |
| mysql-monitor_groupreplication_healthcheck_interval          | 5000                        |
| mysql-monitor_groupreplication_healthcheck_max_timeout_count | 3                           |
| mysql-monitor_groupreplication_healthcheck_timeout           | 800                         |
| mysql-monitor_groupreplication_max_transactions_behind_count | 3                           |
| mysql-monitor_history                                        | 600000                      |
| mysql-monitor_password                                       | Uip.1234!                   |
| mysql-monitor_ping_interval                                  | 10000                       |
| mysql-monitor_ping_max_failures                              | 3                           |
| mysql-monitor_ping_timeout                                   | 1000                        |
| mysql-monitor_query_interval                                 | 60000                       |
| mysql-monitor_query_timeout                                  | 100                         |
| mysql-monitor_read_only_interval                             | 1500                        |
| mysql-monitor_read_only_max_timeout_count                    | 3                           |
| mysql-monitor_read_only_timeout                              | 500                         |
| mysql-monitor_replication_lag_interval                       | 10000                       |
| mysql-monitor_replication_lag_timeout                        | 1000                        |
| mysql-monitor_replication_lag_use_percona_heartbeat          |                             |
| mysql-monitor_slave_lag_when_null                            | 60                          |
| mysql-monitor_threads_max                                    | 128                         |
| mysql-monitor_threads_min                                    | 8                           |
| mysql-monitor_threads_queue_maxsize                          | 128                         |
| mysql-monitor_username                                       | monitor                     |
| mysql-monitor_wait_timeout                                   | true                        |
| mysql-monitor_writer_is_also_reader                          | true                        |
| mysql-multiplexing                                           | true                        |
| mysql-ping_interval_server_msec                              | 120000                      |
| mysql-ping_timeout_server                                    | 500                         |
| mysql-poll_timeout                                           | 2000                        |
| mysql-poll_timeout_on_failure                                | 100                         |
| mysql-query_cache_size_MB                                    | 256                         |
| mysql-query_cache_stores_empty_result                        | true                        |
| mysql-query_digests                                          | true                        |
| mysql-query_digests_lowercase                                | false                       |
| mysql-query_digests_max_digest_length                        | 2048                        |
| mysql-query_digests_max_query_length                         | 65000                       |
| mysql-query_digests_no_digits                                | false                       |
| mysql-query_digests_normalize_digest_text                    | false                       |
| mysql-query_digests_replace_null                             | false                       |
| mysql-query_digests_track_hostname                           | false                       |
| mysql-query_processor_iterations                             | 0                           |
| mysql-query_processor_regex                                  | 1                           |
| mysql-query_retries_on_failure                               | 1                           |
| mysql-reset_connection_algorithm                             | 2                           |
| mysql-server_capabilities                                    | 569899                      |
| mysql-server_version                                         | 8.0.19                      |
| mysql-servers_stats                                          | true                        |
| mysql-session_idle_ms                                        | 1000                        |
| mysql-session_idle_show_processlist                          | true                        |
| mysql-sessions_sort                                          | true                        |
| mysql-set_query_lock_on_hostgroup                            | 1                           |
| mysql-show_processlist_extended                              | 0                           |
| mysql-shun_on_failures                                       | 5                           |
| mysql-shun_recovery_time_sec                                 | 10                          |
| mysql-ssl_p2s_ca                                             |                             |
| mysql-ssl_p2s_cert                                           |                             |
| mysql-ssl_p2s_cipher                                         |                             |
| mysql-ssl_p2s_key                                            |                             |
| mysql-stacksize                                              | 1048576                     |
| mysql-stats_time_backend_query                               | false                       |
| mysql-stats_time_query_processor                             | false                       |
| mysql-tcp_keepalive_time                                     | 0                           |
| mysql-threads                                                | 4                           |
| mysql-threshold_query_length                                 | 524288                      |
| mysql-threshold_resultset_size                               | 4194304                     |
| mysql-throttle_connections_per_sec_to_hostgroup              | 1000000                     |
| mysql-throttle_max_bytes_per_second_to_client                | 0                           |
| mysql-throttle_ratio_server_to_client                        | 0                           |
| mysql-use_tcp_keepalive                                      | 0                           |
| mysql-verbose_query_error                                    | false                       |
| mysql-wait_timeout                                           | 28800000                    |
+--------------------------------------------------------------+-----------------------------+