[Enhancement] Improve the performance of "insert into select" statement

Search before asking

[X] I had searched in the issues and found no similar issues.

Description

Currently, the performence of "insert into select" statement is very poor, doris 1.1.0-rc03 is 37M/s for the cluster 3 BE Node (16c64g), and for the same case, the rate of impala is 100G/s.

The detail as following:

`MySQL [ssb]> insert into lineorder_flat2 select * from lineorder_flat; Query OK, 600037902 rows affected (26 min 52.81 sec) {'label':'insert_2ed2773ca6654469-81283daa905be7c2', 'status':'VISIBLE', 'txnId':'235'}

MySQL [ssb]> show data; +-----------------+-------------+--------------+ | TableName | Size | ReplicaCount | +-----------------+-------------+--------------+ | customer | 138.437 MB | 12 | | dates | 34.228 KB | 1 | | lineorder | 14.611 GB | 336 | | lineorder_flat | 58.854 GB | 336 | | lineorder_flat2 | 58.854 GB | 336 | | part | 12.969 MB | 12 | | supplier | 9.158 MB | 12 | | Total | 132.476 GB | 1045 | | Quota | 1024.000 TB | 1073741824 | | Left | 1023.871 TB | 1073740779 | +-----------------+-------------+--------------+ 10 rows in set (0.01 sec)

Solution

Refer the Implement of Impala rather than stream load? I'm not sure, Maybe Someone has good idea.

Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

apache / doris