apache / cloudberry

One advanced and mature open-source MPP (Massively Parallel Processing) database. Open source alternative to Greenplum Database.
https://cloudberry.apache.org
Apache License 2.0
430 stars 104 forks source link

Support sreh in copy from parse stage #621

Closed weinan003 closed 2 months ago

weinan003 commented 2 months ago

After commit f82de5c46bd, COPY FROM add a parse stage at the very begin process. Our sreh can not handle the error which the data have encoding error.

Since the raw_buf,input_buf and line_buf does not cleanup after parse stage error have raised, even sreh cache the ERROR message, copy infinitly stuck at the error stage.

Fix: if the parse error, in the stage internal, skip the error record by EOL.

fix #ISSUE_Number


Change logs

Describe your change clearly, including what problem is being solved or what feature is being added.

If it has some breaking backward or forward compatibility, please clary.

Why are the changes needed?

Describe why the changes are necessary.

Does this PR introduce any user-facing change?

If yes, please clarify the previous behavior and the change this PR proposes.

How was this patch tested?

Please detail how the changes were tested, including manual tests and any relevant unit or integration tests.

Contributor's Checklist

Here are some reminders and checklists before/when submitting your pull request, please check them:

my-ship-it commented 2 months ago

Please fix CI failures?

weinan003 commented 2 months ago

Please fix CI failures?

Compared with the upstream's answer file, our answer file is this bug we have fixed. update the correct answer file back