Open ozgune opened 8 years ago
I'm adding a note here from our old issue tracker.
"Add a configuration option to skip over a certain number of malformed lines when uploading data. I quite like how the nzload command has this option."
Note: this can probably be achieved with putting a PG_TRY and PG_CATCH around just NextCopyFrom in the COPY implementation.
When users load large data sets (from S3 or files), these datasets might have a few bad records. Most data warehousing solutions can be configured to skip over a predefined number of bad lines.
This has also been discussed for PostgreSQL: https://wiki.postgresql.org/wiki/Error_logging_in_COPY
This task proposes to extend COPY to skip over a configurable number of records.