ApsaraDB / PolarDB-for-PostgreSQL

A cloud-native database based on PostgreSQL developed by Alibaba Cloud.
https://apsaradb.github.io/PolarDB-for-PostgreSQL/zh/
Apache License 2.0
2.89k stars 481 forks source link

increase insert xxx select xxx speed #461

Closed osdba closed 1 month ago

osdba commented 11 months ago

when increase polar_bulk_read_size to 64, but "insert xxx select" not getting faster, because ring_size is low,so increase ring_size in function GetAccessStrategy.

test:

postgres=# reset polar_bulk_read_size;
RESET
Time: 0.200 ms
postgres=# show polar_bulk_read_size;
 polar_bulk_read_size
----------------------
 128kB
(1 row)

Time: 0.216 ms
postgres=# select count(*) from test2;
   count
-----------
 200000000
(1 row)

Time: 96621.768 ms (01:36.622)

postgres=# set polar_bulk_read_size to 64;
SET
postgres=# set max_parallel_workers_per_gather to 0;
SET
postgres=# \timing
Timing is on.
postgres=# select count(*) from test2;
   count
-----------
 200000000
(1 row)

Time: 73271.417 ms (01:13.271)

postgres=# set max_parallel_workers_per_gather to 8;
SET
Time: 0.173 ms
postgres=# explain select count(*) from test2;
                                         QUERY PLAN
---------------------------------------------------------------------------------------------
 Finalize Aggregate  (cost=9009314.43..9009314.44 rows=1 width=8)
   ->  Gather  (cost=9009313.60..9009314.41 rows=8 width=8)
         Workers Planned: 8
         ->  Partial Aggregate  (cost=9008313.60..9008313.61 rows=1 width=8)
               ->  Parallel Seq Scan on test2  (cost=0.00..8945812.48 rows=25000448 width=0)
(5 rows)

Time: 0.454 ms
postgres=# select count(*) from test2;
   count
-----------
 200000000
(1 row)

Time: 92410.190 ms (01:32.410)

postgres=# set polar_bulk_read_size to 64;
SET
Time: 0.168 ms
postgres=# show polar_bulk_read_size;
 polar_bulk_read_size
----------------------
 512kB
(1 row)

Time: 0.154 ms
postgres=# select count(*) from test2;
   count
-----------
 200000000
(1 row)

Time: 75722.028 ms (01:15.722)
polardb-bot[bot] commented 11 months ago

Hi @osdba ~ Thanks for your contribution in this PR. ❤️

Please make sure that your PR conforms the standard, and has passed all the checks.

We will review your PR as soon as possible.

polardb-bot[bot] commented 11 months ago

Hey @osdba :

Congratulations~ 🎉 Your commit has passed all the checks. Please wait for further manual review.

mrdrivingduck commented 1 month ago

/close