search

dimitri / pgloader

Migrate to PostgreSQL in a single command!
http://pgloader.io
Other
5.42k stars 546 forks source link

rejected file dat not refer to original line of imported file #1192

Open esoni opened 4 years ago

esoni commented 4 years ago

Hi All. i m using pgloader to import csv in my postgres db. I m using pgloader 3.6.2. I m importing the following file csv

 a;b;c;d;e;1,000.54
a;b;c;d;e;dfd

without going into the detail of my load file, the second line is correctly rejected by db. Pg loader produces the following dat file that contain the rejected record

a b c d e dfd 201905

Unfortunately the file contains the row after pgloader elaboration ( 201905 is a value calculated using a common lisp expression). From my perspective, as oracle loader, the file should contain the rejected lines as originally read from csv file. in my example

a;b;c;d;e;dfd and not the row after pg loader elaboration. thanks a lot dario

dimitri commented 4 years ago

In the general case it's very hard to do that because CSV input could contain line separator. So the CSV parser would have to keep that information available to us, and I don't think cl-csv does that.