dimitri / pgloader

Migrate to PostgreSQL in a single command!
http://pgloader.io
Other
5.44k stars 547 forks source link

`WITH max parallel create index 1` is still parallelly run many `CREATE INDEX` and the importing of next table #1576

Open n0099 opened 7 months ago

n0099 commented 7 months ago

By periodically running

SELECT * FROM pg_stat_activity WHERE application_name = 'pgloader';

I can see when the COPY for first table is complete there will be multiple(as many as the count of indexes on that table) of CREATE TABLE running and the COPY of next table, this behavior will slow down database and all these queries. Then I tried decreasing the parallelism by max parallel create index 1 https://pgloader.readthedocs.io/en/latest/batches.html

By default, as many CREATE INDEX threads as the maximum number of indexes per table are found in your source schema. It is possible to set the max parallel create index WITH option to another number in case there’s just too many of them to create.

but it's still the same.