`csv header` can't dynamically handle out-of-order columns

[seamusabshere]

• [x] pgloader --version

pgloader version "3.5.1ee389d"
compiled with Clozure Common Lisp Version 1.11.5/v1.11.5  (LinuxX8664)

• [x] did you test a fresh compile from the source tree?

• [x] did you search for other similar issues?

• [x] how can I reproduce the bug?

[skipping some of the steps below because this is a simple "bug"]

basically i have a table like this:

CREATE TABLE foo (
  _c int,
  a int,
  b text
)

and a csv like this

b,a
hello,1
world,2

Notice they have the same column names but different order, and the table has an extra column.

LOAD CSV
FROM '$FILE_PATH'
INTO $DB_URL
WITH
  csv header,
  fields optionally enclosed by '"',
  fields escaped by double-quote,
  fields terminated by ','
SET client_encoding to 'utf8'
;

What I expect: csv header is smart enough to know to import a->a, b->b, and not try to import anything into _c. But it tries to import a->_c (I know because it says Database error 22P02: invalid input syntax for integer etc.)

I know that I can specify a different ordering by providing HAVING FIELDS and TARGET COLUMNS, but shouldn't it be able to do that automatically?

[dimitri]

Yeah it's fair to say that pgloader should be able to handle that case automatically. I've implemented the above in the referenced patch. Please compile from fresh sources and try again, and open this issue again is something is still off.

[seamusabshere]

@dimitri best bug response ever