A large query triggers oom, help

sysown / proxysql

High-performance MySQL proxy with a GPL license.

http://www.proxysql.com

GNU General Public License v3.0

5.87k stars 960 forks source link

A large query triggers oom, help #1990

Open lazzyfu opened 5 years ago

lazzyfu commented 5 years ago

hi: When the query returns a large amount of data, it will trigger the system's OOM。The proxysql process will be killed。 This issue exists in the following versions.

proxysql 1.4.15
proxysql 2.0.3

This should be because proxysql receives data faster than the client consumes. I executed a query with about 20 million data volumes.

As shown below：

I adjusted the parameter mysql-throttle_max_bytes_per_second_to_client and it didn't work.

I read the relevant questions, but did not solve them, ask for advice, thank you

lazzyfu commented 5 years ago

Online, thank you

renecannao commented 5 years ago

Is the query using binary protocol (prepared statement) or text protocol?

lazzyfu commented 5 years ago

hi. He is a go program, should be using prepared statement，thanks.

lazzyfu commented 5 years ago

I am using this package https://github.com/jmoiron/sqlx thanks

renecannao commented 5 years ago

If the client is using prepared statements, currently proxysql cannot throttle traffic. It will fetch all the resultset before sending it to the client.

I think we need to introduce a new variable to define how much data can be buffered before aborting the request.

lazzyfu commented 5 years ago

I am looking forward to this parameter, thanks. Is there an improved plan?

It should be session level

winmutt commented 3 years ago

Can you clarify the limitation that requires the need to return the full dataset?