darold / ora2pg

Ora2Pg is a free tool used to migrate an Oracle database to a PostgreSQL compatible schema. It connects your Oracle database, scan it automatically and extracts its structure or data, it then generates SQL scripts that you can load into PostgreSQL.
http://www.ora2pg.com/
GNU General Public License v3.0
1.03k stars 343 forks source link

migration stops at 5.4 Gb #1213

Closed GerrMullers closed 3 years ago

GerrMullers commented 3 years ago

Hi, I have the next problem, the oracle base for migration is 400GB, made several attempts migration always stops at 5.4GB. Tell me where to look for a solution

darold commented 3 years ago

Is there is any error message? You should check that the OOM Killer was not fire; use dmesg system command to see if ora2pg have been killed.

GerrMullers commented 3 years ago

i have error lice this: DBD::Pg::db pg_putcopyend failed: ERROR: no partition of relation "history" found for row DETAIL: Partition key of the failing row contains (clock) = (1626672322). CONTEXT: COPY history, line 1: "444412 1626672322 45.4689 852735892" at /usr/local/share/perl5/Ora2Pg.pm line 15781. Aborting export... ora2pg in memory 230465 postgre+ 20 0 2532148 174384 5076 R 59.0 1.1 0:01.77 ora2pg - sendin 192408 postgre+ 20 0 2508496 165836 20028 S 11.7 1.0 5:05.59 ora2pg - queryi

darold commented 3 years ago

Well why not starting the issue with this error message, it is much more obvious than your first post. The error message is explicit you have not imported the necessary partitions into your PG database.

GerrMullers commented 3 years ago

I've been unloading tables with this command ora2pg -d -t TABLE -o data.sql -b ./data -c ./config/ora2pg.conf

GerrMullers commented 3 years ago

config parametr DISABLE_PARTITION 1

darold commented 3 years ago

What does \d history executed in psql reports?

GerrMullers commented 3 years ago

zabbdb_dn=# \d history Did not find any relation named "history".

darold commented 3 years ago

well it must not be in the public schema, but look if your history table is partitioned or not.

GerrMullers commented 3 years ago

zabbdb_dn=# \d zabbix.history Partitioned table "zabbix.history" Column | Type | Collation | Nullable | Default --------+---------------+-----------+----------+------------- itemid | numeric(20,0) | | not null | clock | bigint | | not null | '0'::bigint value | numeric(20,4) | | not null | 0.0000 ns | bigint | | not null | '0'::bigint Partition key: RANGE (clock) Indexes: "alfa_history$itemid" btree (itemid) "history_1" btree (itemid, clock) Number of partitions: 0

darold commented 3 years ago

Then you have a partitioned table but no partition defined. I you don't want to reproduce the partitioning you must enable DISABLE_PARTITION when exporting table. Verify the output file you might have a PARTITION BY clause.

GerrMullers commented 3 years ago

I want to try both options

GerrMullers commented 3 years ago

I have 4 partitioned table in my database. what i need in conf file to reproduce the partitioning.

darold commented 3 years ago

Disable DISABLE_PARTITION and export using PARTITION action this might be what you are looking for.

GerrMullers commented 3 years ago

ora2pg -d -t PARTITION TABLE -o data_part.sql -b ./data -c ./config/ora2pg.conf

I'm right ?

GerrMullers commented 3 years ago

is it possible to skip some of the PARTITION during the transfer, and transfer them later

darold commented 3 years ago

See EXCLUDE directive.