search

StarRocks / starrocks

StarRocks, a Linux Foundation project, is a next-generation sub-second MPP OLAP database for full analytics scenarios, including multi-dimensional analytics, real-time analytics, and ad-hoc queries.
https://starrocks.io
Apache License 2.0
8.75k stars 1.75k forks source link

[array] fe lost connection when array_agg large data #44645

Open wangruin opened 5 months ago

wangruin commented 5 months ago

Steps to reproduce the behavior (Required)

use ssb 1g data, 600w rows

mysql> select array_agg(lo_orderdate) from lineorder_flat;
ERROR 2020 (HY000): Got packet bigger than 'max_allowed_packet' bytes
mysql> select array_agg(lo_orderdate) from lineorder_flat;
ERROR 2013 (HY000): Lost connection to MySQL server during query
No connection. Trying to reconnect...
Connection id:    29
Current database: ssb

ERROR 2020 (HY000): Got packet bigger than 'max_allowed_packet' bytes

Expected behavior (Required)

query error, not lost connection

Real behavior (Required)

2024-04-24 11:02:20.320+08:00 WARN (starrocks-mysql-nio-pool-10|253) [PruneComplexTypeUtil$Context.checkCanPrune():171] Complex type columnRefOperator[ARRAY<DATE>] and scalarOperator[struct<col1 array<date>>] should has the same type
2024-04-24 11:02:21.759+08:00 WARN (starrocks-mysql-nio-pool-10|253) [StmtExecutor.execute():682] execute IOException
java.io.IOException: Broken pipe
    at sun.nio.ch.FileDispatcherImpl.write0(Native Method) ~[?:?]
    at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) ~[?:?]
    at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:113) ~[?:?]
    at sun.nio.ch.IOUtil.write(IOUtil.java:79) ~[?:?]
    at sun.nio.ch.IOUtil.write(IOUtil.java:50) ~[?:?]
    at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:462) ~[?:?]
    at org.xnio.nio.NioSocketConduit.write(NioSocketConduit.java:153) ~[xnio-nio-3.8.10.Final.jar:3.8.10.Final]
    at org.xnio.conduits.ConduitStreamSinkChannel.write(ConduitStreamSinkChannel.java:150) ~[xnio-api-3.8.10.Final.jar:3.8.10.Final]
    at org.xnio.channels.Channels.writeBlocking(Channels.java:147) ~[xnio-api-3.8.10.Final.jar:3.8.10.Final]
    at com.starrocks.mysql.nio.NMysqlChannel.realNetSend(NMysqlChannel.java:66) ~[starrocks-fe.jar:?]
    at com.starrocks.mysql.MysqlChannel.send(MysqlChannel.java:236) ~[starrocks-fe.jar:?]
    at com.starrocks.mysql.MysqlChannel.writeBuffer(MysqlChannel.java:301) ~[starrocks-fe.jar:?]
    at com.starrocks.mysql.MysqlChannel.sendOnePacket(MysqlChannel.java:316) ~[starrocks-fe.jar:?]
    at com.starrocks.qe.StmtExecutor.handleQueryStmt(StmtExecutor.java:1044) ~[starrocks-fe.jar:?]
    at com.starrocks.qe.StmtExecutor.execute(StmtExecutor.java:578) ~[starrocks-fe.jar:?]
    at com.starrocks.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:421) ~[starrocks-fe.jar:?]
    at com.starrocks.qe.ConnectProcessor.dispatch(ConnectProcessor.java:617) ~[starrocks-fe.jar:?]
    at com.starrocks.qe.ConnectProcessor.processOnce(ConnectProcessor.java:921) ~[starrocks-fe.jar:?]
    at com.starrocks.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:69) ~[starrocks-fe.jar:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
    at java.lang.Thread.run(Thread.java:829) ~[?:?]
2024-04-24 11:02:21.760+08:00 WARN (starrocks-mysql-nio-pool-10|253) [DefaultCoordinator.cancel():831] cancel execState of query, this is outside invoke
2024-04-24 11:02:21.760+08:00 WARN (starrocks-mysql-nio-pool-10|253) [ReadListener.lambda$handleEvent$0():81] Exception happened in one session(com.starrocks.mysql.nio.NConnectContext@4b6455d8).
java.io.IOException: Broken pipe
    at sun.nio.ch.FileDispatcherImpl.write0(Native Method) ~[?:?]
    at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) ~[?:?]
    at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:113) ~[?:?]
    at sun.nio.ch.IOUtil.write(IOUtil.java:79) ~[?:?]
    at sun.nio.ch.IOUtil.write(IOUtil.java:50) ~[?:?]
    at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:462) ~[?:?]
    at org.xnio.nio.NioSocketConduit.write(NioSocketConduit.java:153) ~[xnio-nio-3.8.10.Final.jar:3.8.10.Final]
    at org.xnio.conduits.ConduitStreamSinkChannel.write(ConduitStreamSinkChannel.java:150) ~[xnio-api-3.8.10.Final.jar:3.8.10.Final]
    at org.xnio.channels.Channels.writeBlocking(Channels.java:147) ~[xnio-api-3.8.10.Final.jar:3.8.10.Final]
    at com.starrocks.mysql.nio.NMysqlChannel.realNetSend(NMysqlChannel.java:66) ~[starrocks-fe.jar:?]
    at com.starrocks.mysql.MysqlChannel.send(MysqlChannel.java:236) ~[starrocks-fe.jar:?]
    at com.starrocks.mysql.MysqlChannel.flush(MysqlChannel.java:259) ~[starrocks-fe.jar:?]
    at com.starrocks.mysql.MysqlChannel.sendAndFlush(MysqlChannel.java:327) ~[starrocks-fe.jar:?]
    at com.starrocks.qe.ConnectProcessor.finalizeCommand(ConnectProcessor.java:713) ~[starrocks-fe.jar:?]
    at com.starrocks.qe.ConnectProcessor.processOnce(ConnectProcessor.java:923) ~[starrocks-fe.jar:?]
    at com.starrocks.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:69) ~[starrocks-fe.jar:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
    at java.lang.Thread.run(Thread.java:829) ~[?:?]
2024-04-24 11:02:23.225+08:00 WARN (starrocks-mysql-nio-pool-10|253) [PruneComplexTypeUtil$Context.checkCanPrune():171] Complex type columnRefOperator[ARRAY<DATE>] and scalarOperator[struct<col1 array<date>>] should has the same type
2024-04-24 11:02:24.749+08:00 WARN (starrocks-mysql-nio-pool-10|253) [StmtExecutor.execute():682] execute IOException
java.io.IOException: Broken pipe
    at sun.nio.ch.FileDispatcherImpl.write0(Native Method) ~[?:?]
    at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) ~[?:?]
    at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:113) ~[?:?]
    at sun.nio.ch.IOUtil.write(IOUtil.java:79) ~[?:?]
    at sun.nio.ch.IOUtil.write(IOUtil.java:50) ~[?:?]
    at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:462) ~[?:?]
    at org.xnio.nio.NioSocketConduit.write(NioSocketConduit.java:153) ~[xnio-nio-3.8.10.Final.jar:3.8.10.Final]
    at org.xnio.conduits.ConduitStreamSinkChannel.write(ConduitStreamSinkChannel.java:150) ~[xnio-api-3.8.10.Final.jar:3.8.10.Final]
    at org.xnio.channels.Channels.writeBlocking(Channels.java:147) ~[xnio-api-3.8.10.Final.jar:3.8.10.Final]
    at com.starrocks.mysql.nio.NMysqlChannel.realNetSend(NMysqlChannel.java:66) ~[starrocks-fe.jar:?]
    at com.starrocks.mysql.MysqlChannel.send(MysqlChannel.java:236) ~[starrocks-fe.jar:?]
    at com.starrocks.mysql.MysqlChannel.writeBuffer(MysqlChannel.java:301) ~[starrocks-fe.jar:?]
    at com.starrocks.mysql.MysqlChannel.sendOnePacket(MysqlChannel.java:316) ~[starrocks-fe.jar:?]
    at com.starrocks.qe.StmtExecutor.handleQueryStmt(StmtExecutor.java:1044) ~[starrocks-fe.jar:?]
    at com.starrocks.qe.StmtExecutor.execute(StmtExecutor.java:578) ~[starrocks-fe.jar:?]
    at com.starrocks.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:421) ~[starrocks-fe.jar:?]
    at com.starrocks.qe.ConnectProcessor.dispatch(ConnectProcessor.java:617) ~[starrocks-fe.jar:?]
    at com.starrocks.qe.ConnectProcessor.processOnce(ConnectProcessor.java:921) ~[starrocks-fe.jar:?]
    at com.starrocks.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:69) ~[starrocks-fe.jar:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
    at java.lang.Thread.run(Thread.java:829) ~[?:?]
2024-04-24 11:02:24.749+08:00 WARN (starrocks-mysql-nio-pool-10|253) [DefaultCoordinator.cancel():831] cancel execState of query, this is outside invoke
2024-04-24 11:02:24.749+08:00 WARN (starrocks-mysql-nio-pool-10|253) [ReadListener.lambda$handleEvent$0():81] Exception happened in one session(com.starrocks.mysql.nio.NConnectContext@5303e905).
java.io.IOException: Broken pipe
    at sun.nio.ch.FileDispatcherImpl.write0(Native Method) ~[?:?]
    at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) ~[?:?]
    at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:113) ~[?:?]
    at sun.nio.ch.IOUtil.write(IOUtil.java:79) ~[?:?]
    at sun.nio.ch.IOUtil.write(IOUtil.java:50) ~[?:?]
    at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:462) ~[?:?]
    at org.xnio.nio.NioSocketConduit.write(NioSocketConduit.java:153) ~[xnio-nio-3.8.10.Final.jar:3.8.10.Final]
    at org.xnio.conduits.ConduitStreamSinkChannel.write(ConduitStreamSinkChannel.java:150) ~[xnio-api-3.8.10.Final.jar:3.8.10.Final]
    at org.xnio.channels.Channels.writeBlocking(Channels.java:147) ~[xnio-api-3.8.10.Final.jar:3.8.10.Final]
    at com.starrocks.mysql.nio.NMysqlChannel.realNetSend(NMysqlChannel.java:66) ~[starrocks-fe.jar:?]
    at com.starrocks.mysql.MysqlChannel.send(MysqlChannel.java:236) ~[starrocks-fe.jar:?]
    at com.starrocks.mysql.MysqlChannel.flush(MysqlChannel.java:259) ~[starrocks-fe.jar:?]
    at com.starrocks.mysql.MysqlChannel.sendAndFlush(MysqlChannel.java:327) ~[starrocks-fe.jar:?]
    at com.starrocks.qe.ConnectProcessor.finalizeCommand(ConnectProcessor.java:713) ~[starrocks-fe.jar:?]
    at com.starrocks.qe.ConnectProcessor.processOnce(ConnectProcessor.java:923) ~[starrocks-fe.jar:?]
    at com.starrocks.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:69) ~[starrocks-fe.jar:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
    at java.lang.Thread.run(Thread.java:829) ~[?:?]

StarRocks version (Required)

Seaven commented 5 months ago

mysql client doesn't support large data read