search

osalvador / ReplicaDB

ReplicaDB is open source tool for database replication, designed for efficiently transferring bulk data between relational and non-relational databases
https://osalvador.github.io/ReplicaDB/
Apache License 2.0
405 stars 97 forks source link

Oracle to postgres data sync issue #8

Open ud20 opened 4 years ago

ud20 commented 4 years ago

We are trying to do a data dump from oracle to postgres. On some larger tables(>120GB) we are seeing some errors. ERROR ReplicaTask:72 ERROR in TaskId-0 inserting data to sink table: ORA-01555: snapshot too old: rollback segment number 20 with name "_SYSSMU20_231399234$" too small We tried a few different things like, do it when the data is not changing constantly, smaller fetch size, less number of jobs running but, nothing seemed to work so far. Did you ever see anything similar happening?

osalvador commented 4 years ago

Hello, ud20,

This is a common error with Oracle with large tables. Check if you can make any of the recommendations listed on this website: https://blog.enmotech.com/2018/09/10/ora-01555-snapshot-old-error-ways-to-avoid-ora-01555-snapshot-too-old-error/

In parallel I am investigating how to address it directly with ReplicaDB.

Greetings

ud20 commented 4 years ago

Hello @osalvador We did it in smaller chunks. 2 years of data at once and finished the migration. Thank you for your help!

osalvador commented 4 years ago

Thanks to you @ud20!

I keep the issue open until I find a solution to this problem directly in ReplicaDB.

Regards!