sysown / proxysql

High-performance MySQL proxy with a GPL license.
http://www.proxysql.com
GNU General Public License v3.0
6.01k stars 978 forks source link

Querying admin fails using MySQL JDBC connector #1671

Open arawak opened 6 years ago

arawak commented 6 years ago

I'm trying to use JDBC (Spring Boot) to access the admin port on ProxySQL 1.4.10, but I get a failure when I issue the query "select * from mysql_users":

java.sql.SQLException: unrecognized token: "@"
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:965) ~[mysql-connector-java-5.1.46.jar:5.1.46]
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3976) ~[mysql-connector-java-5.1.46.jar:5.1.46]
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3912) ~[mysql-connector-java-5.1.46.jar:5.1.46]
    at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2530) ~[mysql-connector-java-5.1.46.jar:5.1.46]
    at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2683) ~[mysql-connector-java-5.1.46.jar:5.1.46]
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2482) ~[mysql-connector-java-5.1.46.jar:5.1.46]

The source code is here: https://github.com/ArawakCC/proxysql-k8s-agent

Basically I'm running ProxySQL cluster with three instances inside a namespace in Kubernetes (minikube). My project is intended to provide a way to change ProxySQL configuration via an HTTP API.

When I hit my /users endpoint, it issues "select * from mysql_users" which invokes the connection pool, creates a connection, and sends the preamble SELECT, which ProxySQL doesn't recognise and rejects.

TCPdump:

sean@mitchells-lt:~/scratch$ sudo tcpdump -A -i vboxnet0 port 30617 -v
tcpdump: listening on vboxnet0, link-type EN10MB (Ethernet), capture size 262144 bytes
08:54:24.094373 IP (tos 0x0, ttl 64, id 50029, offset 0, flags [DF], proto TCP (6), length 60)
    mitchells-lt.39368 > blue.minikube.30617: Flags [S], cksum 0x4f2b (correct), seq 2769071566, win 29200, options [mss 1460,sackOK,TS val 1718842461 ecr 0,nop,wscale 7], length 0
E..<.m@.@./...c...cd..w...........r.O+.........
fst]........
08:54:24.095135 IP (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 60)
    blue.minikube.30617 > mitchells-lt.39368: Flags [S.], cksum 0xbd14 (correct), seq 2258602644, ack 2769071567, win 28960, options [mss 1460,sackOK,TS val 2097349822 ecr 1718842461,nop,wscale 7], length 0
E..<..@.?.....cd..c.w.............q ...........
}...fst]....
08:54:24.095168 IP (tos 0x0, ttl 64, id 50030, offset 0, flags [DF], proto TCP (6), length 52)
    mitchells-lt.39368 > blue.minikube.30617: Flags [.], cksum 0x5c1c (correct), ack 1, win 229, options [nop,nop,TS val 1718842461 ecr 2097349822], length 0
E..4.n@.@./...c...cd..w.............\......
fst]}...
08:54:24.096064 IP (tos 0x0, ttl 63, id 21528, offset 0, flags [DF], proto TCP (6), length 130)
    blue.minikube.30617 > mitchells-lt.39368: Flags [P.], cksum 0xba8e (correct), seq 1:79, ack 1, win 227, options [nop,nop,TS val 2097349823 ecr 1718842461], length 78
E...T.@.?.....cd..c.w......................
}...fst]J...
5.5.30../..cn#^0stl.*.!...............AAaDq-+/H^#-.mysql_native_password.
08:54:24.096089 IP (tos 0x0, ttl 64, id 50031, offset 0, flags [DF], proto TCP (6), length 52)
    mitchells-lt.39368 > blue.minikube.30617: Flags [.], cksum 0x5bcc (correct), ack 79, win 229, options [nop,nop,TS val 1718842462 ecr 2097349823], length 0
E..4.o@.@./...c...cd..w.............[......
fst^}...
08:54:24.096921 IP (tos 0x0, ttl 64, id 50032, offset 0, flags [DF], proto TCP (6), length 145)
    mitchells-lt.39368 > blue.minikube.30617: Flags [P.], cksum 0xdabf (correct), seq 1:94, ack 79, win 229, options [nop,nop,TS val 1718842463 ecr 2097349823], length 93
E....p@.@./@..c...cd..w....................
fst_}...Y.....
.....!.......................cluster1..~....x.y.O...9....\.main.mysql_native_password.
08:54:24.097491 IP (tos 0x0, ttl 63, id 21529, offset 0, flags [DF], proto TCP (6), length 52)
    blue.minikube.30617 > mitchells-lt.39368: Flags [.], cksum 0x5b6f (correct), ack 94, win 227, options [nop,nop,TS val 2097349824 ecr 1718842463], length 0
E..4T.@.?.....cd..c.w..........,....[o.....
}...fst_
08:54:24.097614 IP (tos 0x0, ttl 63, id 21530, offset 0, flags [DF], proto TCP (6), length 63)
    blue.minikube.30617 > mitchells-lt.39368: Flags [P.], cksum 0x5259 (correct), seq 79:90, ack 94, win 227, options [nop,nop,TS val 2097349825 ecr 1718842463], length 11
E..?T.@.?.....cd..c.w..........,....RY.....
}...fst_...........
08:54:24.097890 IP (tos 0x0, ttl 64, id 50033, offset 0, flags [DF], proto TCP (6), length 980)
    mitchells-lt.39368 > blue.minikube.30617: Flags [P.], cksum 0x405c (correct), seq 94:1022, ack 90, win 229, options [nop,nop,TS val 1718842464 ecr 2097349825], length 928
E....q@.@.+...c...cd..w....,........@\.....
fst`}......../* mysql-connector-java-5.1.46 ( Revision: 9cc87a48e75c2d2e87c1a293b2862ce651cb256e ) */SELECT  @@session.auto_increment_increment AS auto_increment_increment, @@character_set_client AS character_set_client, @@character_set_connection AS character_set_connection, @@character_set_results AS character_set_results, @@character_set_server AS character_set_server, @@collation_server AS collation_server, @@init_connect AS init_connect, @@interactive_timeout AS interactive_timeout, @@license AS license, @@lower_case_table_names AS lower_case_table_names, @@max_allowed_packet AS max_allowed_packet, @@net_buffer_length AS net_buffer_length, @@net_write_timeout AS net_write_timeout, @@query_cache_size AS query_cache_size, @@query_cache_type AS query_cache_type, @@sql_mode AS sql_mode, @@system_time_zone AS system_time_zone, @@time_zone AS time_zone, @@tx_isolation AS transaction_isolation, @@wait_timeout AS wait_timeout
08:54:24.098510 IP (tos 0x0, ttl 63, id 21531, offset 0, flags [DF], proto TCP (6), length 88)
    blue.minikube.30617 > mitchells-lt.39368: Flags [P.], cksum 0x69f5 (correct), seq 90:126, ack 1022, win 241, options [nop,nop,TS val 2097349826 ecr 1718842464], length 36
E..XT.@.?.....cd..c.w...............i......
}...fst` ......##2800unrecognized token: "@"
08:54:24.098777 IP (tos 0x0, ttl 64, id 50034, offset 0, flags [DF], proto TCP (6), length 52)
    mitchells-lt.39368 > blue.minikube.30617: Flags [F.], cksum 0x5799 (correct), seq 1022, ack 126, win 229, options [nop,nop,TS val 1718842465 ecr 2097349826], length 0
E..4.r@.@./...c...cd..w.............W......
fsta}...
08:54:24.099485 IP (tos 0x0, ttl 63, id 21532, offset 0, flags [DF], proto TCP (6), length 52)
    blue.minikube.30617 > mitchells-lt.39368: Flags [F.], cksum 0x578b (correct), seq 126, ack 1023, win 241, options [nop,nop,TS val 2097349827 ecr 1718842465], length 0
E..4T.@.?.....cd..c.w...............W......
}...fsta
08:54:24.099510 IP (tos 0x0, ttl 64, id 50035, offset 0, flags [DF], proto TCP (6), length 52)
    mitchells-lt.39368 > blue.minikube.30617: Flags [.], cksum 0x5796 (correct), ack 127, win 229, options [nop,nop,TS val 1718842466 ecr 2097349827], length 0
E..4.s@.@./...c...cd..w.............W......
fstb}...

No problem using the CLI.

I also tried using the MySQL driver directly, outside of the connection pool, with the same results. Tried also the Mariadb JDBC connector, which fails differently ("could not load system variables. socket connected: true")

There are similar issues already entered, but the most relevant are closed, and one suggests that a hacked JDBC connector is needed (I did try Drizzle, with no luck).

mrfaiz commented 5 years ago

Hello, Did you find any solution? I am facing same problem. ProxSQL version is 2.0.2

arawak commented 5 years ago

Sorry, no. Didn't really have time to work around it so I decide not to go with ProxySQL for my use case.

If you are in a position to use other stacks than Java, I'm guessing that the Python drivers will work, since there's an Ansible module.

renecannao commented 5 years ago

ProxySQL's Admin is not a MySQL backend. ProxySQL's Admin uses the MySQL protocol so it allows clients and drivers to use this protocol to communicate with it. Although, if the client/driver makes the strong assumption that the backend is a MySQL server and doesn't handle errors, the client/driver will fail. In the case of JDBC, it asks a bunch of variables using @@ (SELECT @@session.auto_increment_increment AS auto_increment_increment, @@character_set_client AS character_set_client, ... ) that do not exist in SQLite. ProxySQL's Admin returns an error, and the JDBC driver gives up.

mrfaiz commented 5 years ago

@arawak thanks .. I will try Python drivers. @renecannao Is there any way for Java ? (like mariadb-connector-j or other)

tjlee commented 5 years ago

@mrfaiz I found a hack workaround. You need to change mysql-server_version to any version grater than 8.0.3. On ProxySQL Admin node execute as admin:

update global_variables set variable_value="8.0.4 (ProxySQL)" 
where variable_name='mysql-server_version';

load mysql variables to run;save mysql variables to disk;

based on MySQL Connector/J source code

tjlee commented 5 years ago

+1 Possible solution for ProxySQL is to set ProxySQL version according to MySQL version.

mdefanti commented 5 years ago

I'm having the same problem with MySQL Connector version 5.1.40 and ProxySQL version 2.0.7. I tried changing the mysql-server_version to different values but none have worked.

Is there a workaround for this ?

petitajt commented 4 years ago

@mrfaiz I found a hack workaround. You need to change mysql-server_version to any version grater than 8.0.3. On ProxySQL Admin node execute as admin:

update global_variables set variable_value="8.0.4 (ProxySQL)" 
where variable_name='mysql-server_version';

load mysql variables to run;save mysql variables to disk;

based on MySQL Connector/J source code

I also tried your workaround with several versions of MySQL Connector/J. I also tried various version of MariaDB connector. All tests were made with a ProxySQL v2.0.8 and a spring boot application (tests done with both Hikari and DBCP2).

Could you confirm that you got this to work and share the versions of the library you used?

Thanks

tjlee commented 4 years ago

@petitajt I was working with MySQL 8.0.17 & ProxySQL 2.0.6

petitajt commented 4 years ago

@tjlee Ok thanks. I finally opted for a trimmed version of MariaDB connector. Mainly removed transaction handling and some options. Not really suitable for heavy prod use but sufficient for my needs.

hylitjob commented 4 years ago

@tjlee Ok thanks. I finally opted for a trimmed version of MariaDB connector. Mainly removed transaction handling and some options. Not really suitable for heavy prod use but sufficient for my needs.

java.sql.SQLException: unrecognized token: "@"

How do you solve the question above with mysql jdbc driver

Thanks!

hylitjob commented 4 years ago

@tjlee Ok thanks. I finally opted for a trimmed version of MariaDB connector. Mainly removed transaction handling and some options. Not really suitable for heavy prod use but sufficient for my needs.

I tried another way to solve this issue Switch the jdbc connector to sqlite3 driver, as follow:

`spring.datasource.url=jdbc:sqlite:/var/lib/proxysql/proxysql.db

spring.datasource.username=

spring.datasource.password=

spring.datasource.driver-class-name=org.sqlite.JDBC`

Update the proxysql config using java mybatis as follow:

`

<select id="findMysqlUserCount" parameterType="java.lang.String"
    resultType="java.lang.Integer">
    select count(0) from mysql_users where comment = #{id}
</select>

<select id="findMysqlServerCount"
    parameterType="java.lang.String" resultType="java.lang.Integer">
    select count(0) from
    mysql_servers where comment = #{id}
</select>

<select id="findMysqlQueryRuleCount"
    parameterType="java.lang.String" resultType="java.lang.Integer">
    select count(0) from
    mysql_query_rules where comment = #{id}
</select>

<insert id="addMysqlUser" parameterType="MysqlUsers">
    insert into mysql_users(
    username,
    password,
    default_hostgroup,
    default_schema,
    active,
    transaction_persistent,
    frontend,backend,
    max_connections,
    schema_locked,
    fast_forward,
    comment)
    values(
    #{username},
    #{password},
    #{defaultHostgroup},
    #{defaultSchema},
    #{active},
    #{transactionPersistent},
    #{frontend},
    #{backend},
    #{maxConnections},
    #{schemaLocked},
    #{fastForward},
    #{id}
    )
</insert>

<update id="updatMysqlUser" parameterType="MysqlUsers">
    update mysql_users set
    username= #{username},
    password= #{password},
    default_hostgroup=#{defaultHostgroup},
    default_schema=#{defaultSchema},
    active= #{active},
    transaction_persistent=#{transactionPersistent},
    frontend= #{frontend},
    backend= #{backend},
    max_connections= #{maxConnections},
    schema_locked= #{schemaLocked},
    fast_forward= #{fastForward}
    where
    comment=#{id}
</update>

`

Then , load disk to mem and runtime as follow, and it works: `public class ProxysqlCommandExecutor {

private static final Logger LOGGER = LoggerFactory.getLogger(ProxysqlCommandExecutor.class);

private static final String[] RESTART_COMMAND_ARRAY = { "systemctl restart proxysql" };
private static final String COMMAND_PREFIX = " mysql -u admin -padmin -h 127.0.0.1  -P6032 --prompt='Admin> ' -e ";

private static final String[] FLUSH_COMMAND_ARRAY = {
        COMMAND_PREFIX + "'LOAD MYSQL VARIABLES TO MEM;'",
        COMMAND_PREFIX + "'LOAD MYSQL SERVERS TO MEM;'", 
        COMMAND_PREFIX + "'LOAD MYSQL USERS TO MEM;'",
        COMMAND_PREFIX + "'LOAD MYSQL QUERY RULES TO MEM;'",
        COMMAND_PREFIX + "'LOAD MYSQL VARIABLES TO RUNTIME;'", 
        COMMAND_PREFIX + "'LOAD MYSQL SERVERS TO RUNTIME;'",
        COMMAND_PREFIX + "'LOAD MYSQL USERS TO RUNTIME;'",
        COMMAND_PREFIX + "'LOAD MYSQL QUERY RULES TO RUNTIME;'" };

public int configFlush() {
    int flag = 0;
    try {
        for (String command : FLUSH_COMMAND_ARRAY) {
            if (this.runCommand(command) > 0) {
                flag = 1;
            }
        }
    } catch (Exception e) {
        e.printStackTrace();
        LOGGER.error("ProxysqlCommandExecutor-configFlush-error[" + e);
    }
    return flag;
}

public int configRefresh(List<String> refreshSqlList) {
    configFlush();
    return 0;
}

public int configRefreshAll() {
    configFlush();
    return 0;
}`