pgloader version "3.6.af8c3c1"
compiled with SBCL 2.1.11.debian
[x] did you test a fresh compile from the source tree?
[x] did you search for other similar issues?
[x] how can I reproduce the bug?
LOAD DATABASE
FROM mysql://user:@localhost/tbm
INTO pgsql://user:@localhost/tbm
WITH quote identifiers, max parallel create index 1
;
[x] pgloader output you obtain
2024-04-12T22:17:18.008001+08:00 LOG pgloader version "3.6.af8c3c1"
2024-04-12T22:17:18.092001+08:00 LOG Migrating from #<MYSQL-CONNECTION mysql://user@localhost:3306/tbm {1007F8F823}>
2024-04-12T22:17:18.092001+08:00 LOG Migrating into #<PGSQL-CONNECTION pgsql://user@localhost:5432/tbm {1007F90443}>
2024-04-12T22:17:21.728043+08:00 LOG report summary reset
By periodically running
SELECT * FROM pg_stat_activity WHERE application_name = 'pgloader';
I can see when the COPY for first table is complete there will be multiple(as many as the count of indexes on that table) of CREATE TABLE running and the COPY of next table, this behavior will slow down database and all these queries.
Then I tried decreasing the parallelism by max parallel create index 1https://pgloader.readthedocs.io/en/latest/batches.html
By default, as many CREATE INDEX threads as the maximum number of indexes per table are found in your source schema. It is possible to set the max parallel create index WITH option to another number in case there’s just too many of them to create.
[x] how can I reproduce the bug?
By periodically running
I can see when the
COPY
for first table is complete there will be multiple(as many as the count of indexes on that table) ofCREATE TABLE
running and theCOPY
of next table, this behavior will slow down database and all these queries. Then I tried decreasing the parallelism bymax parallel create index 1
https://pgloader.readthedocs.io/en/latest/batches.htmlbut it's still the same.