search

dimitri / pgloader

Migrate to PostgreSQL in a single command!
http://pgloader.io
Other
5.47k stars 551 forks source link

Redshift -> Postgres support #806

Closed tendres closed 6 years ago

tendres commented 6 years ago

With the news that pgloader now supports loading data from Postgres to Redshift, the one feature that is lacking is the ability to migrate from Redshift to Postgres. All of the tools in the wild are for getting data into Redshift but not out. Hoping to change that! An environment has been set up on AWS using an EC2 instance for the binary, an RDS Postgres instance, and a Redshift instance for development work in this area. This ticket was created to track progress.

dimitri commented 6 years ago

So pgloader now supports loading to Redshift, that's true, but not from PostgreSQL yet. I think it's time to make this happen though.

About migrating out of Redshift, it only supports COPY FROM, so we would do that with a SELECT statement that goes over all the rows in there. If we add support for PostgreSQL as a source with using a SELECT query rather than COPY, then we should be able to support both the source types at once., Will have a look.

dimitri commented 6 years ago

Quick update: it's now possible to load data from PostgreSQL with pgloader, thanks to https://github.com/dimitri/pgloader/commit/fc3a1949f74bbfbebdbc023d6e55dc15e5d6df33. We still have to add specific “dumbing-down” of pgloader when the source is Redshift, much as we did already when the target is Redshift. That said, it might work already?

dimitri commented 6 years ago

Closing this issue now that we have the basics. The current issue about Redshift support has moved to https://github.com/dimitri/pgloader/issues/860 anyway it seems.