dimitri / pgloader

Migrate to PostgreSQL in a single command!
http://pgloader.io
Other
5.45k stars 548 forks source link

Can't use 'INCLUDING ONLY TABLE NAMES' and 'ALTER SCHEMA...RENAME TO' together. #1490

Open ns-mkusper opened 1 year ago

ns-mkusper commented 1 year ago
pgloader version "3.6.cc2ce6e"
compiled with SBCL 2.1.1.debian

I compiled the latest source code with SBCL.

LOAD DATABASE
     FROM      postgresql://before:before@localhost:5456/before
     INTO      postgresql://after:after@localhost:5455/after

INCLUDING ONLY TABLE NAMES MATCHING ~/./ IN SCHEMA 'before'

ALTER SCHEMA 'before' rename to 'after'

;
» pgloader test-issue.load 

2023-05-18T23:17:07.006068-05:00 LOG pgloader version "3.6.cc2ce6e"
2023-05-18T23:17:07.007159-05:00 LOG Data errors in '/private/tmp/pgloader/'
2023-05-18T23:17:07.007188-05:00 LOG Parsing commands from file #P"/Users/mkusper/db-migration/test-issue.load"
2023-05-18T23:17:07.151934-05:00 LOG Migrating from #<PGSQL-CONNECTION pgsql://before@localhost:5456/before {700C72B103}>
2023-05-18T23:17:07.152016-05:00 LOG Migrating into #<PGSQL-CONNECTION pgsql://after@localhost:5455/after {700C72B653}>
KABOOM!
TYPE-ERROR: The value
              NIL
            is not of type
              PGLOADER.CATALOG:COLUMN
An unhandled error condition has been signalled:
   The value
     NIL
   is not of type
     PGLOADER.CATALOG:COLUMN

What I am doing here?

The value
  NIL
is not of type
  PGLOADER.CATALOG:COLUMN

I added an example dump I am able to reproduce this with in a gist, but it seems pretty unrelated to the data.

The above is just a test scenario which we can use to reproduce the issue easily, but I'd like to pair 'INCLUDING ONLY TABLE NAMES' and 'ALTER SCHEMA...RENAME TO' to be able to migrate many individual schemas from a few databases which each have dozens of their own schemas I am looking to separate. If these two options worked together and had the intended effect of taking the 'before' schema's tables and renaming it to 'after' (and also allowing me to cast to specific types within those schemas) that would be great, but it seems there is a bug preventing pgloader to reason about the schema in the catalog when both of these are used together.