search

citusdata / citus

Distributed PostgreSQL as an extension
https://www.citusdata.com
GNU Affero General Public License v3.0
10.46k stars 662 forks source link

Adaptive Executor throws Out Of Memory error when pulling a lot of records to coordinator #3476

Closed saicitus closed 4 years ago

saicitus commented 4 years ago

pg_dump on 300GB database results in the below error:

pg_dump: error: Dumping the contents of table "my_table" failed: PQgetResult() failed.
pg_dump: error: Error message from server: ERROR:  out of memory
DETAIL:  Failed on request of size 624 in memory context "ExecutorState".

INSERT INTO SELECT via coordinator also throws Out Of Memory sometimes

SELECT that pulls a lot of records (~20 million records) to the coordinator node also throws OOM.

Workaround is to change the executor type to real-time and the OOM doesn't occur.

marcocitus commented 4 years ago

Probably fixed by https://github.com/citusdata/citus/pull/3114

saicitus commented 4 years ago

@marcocitus Should this fix be a part of 9.0 on Hyperscale (Citus)

marcocitus commented 4 years ago

It is part of 9.0.1

onderkalaci commented 4 years ago

@saicitus should we close this? Have you seen this on >9.0.1 ?

marcocitus commented 4 years ago

Closing until we see it on newer versions.