cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
29.86k stars 3.77k forks source link

Import continue with error rows #53177

Open benny-wangaimin opened 4 years ago

benny-wangaimin commented 4 years ago

@mattcrdb

Is your feature request related to a problem? Please describe. When I use import into statement to load some data into table, if the csv file has some erros rows, import will failed ,even the good ones can't be insert into table.

Describe the solution you'd like If csv file has some erros rows, I expect import can handle the exception, good rows will be insert into table and WITH experimental_save_rejected=uri://path/to/save to save errors rows into file. Import into command has one option, such as WITH errors_rows=continue like WITH experimental_save_rejected.

Describe alternatives you've considered we can give WITH errors_rows two choice,one is continue and another is exit. Continue means load good ones into table. Exit means import will exit when meet error.

Additional context ERROR: nodelocal://1/KCFP_CFTYDM.csv: error parsing row 753: expected 15 fields, got 16 (row: “9999,1001003800,BANCA POPOLARE DI MILANO SCARL , ITALY,10,NULL,NULL,0,NULL,01,0,0,0,0,0,0”)

Epic CRDB-7909

Jira issue: CRDB-3893

blathers-crl[bot] commented 4 years ago

Hello, I am Blathers. I am here to help you get the issue triaged.

It looks like you have not filled out the issue in the format of any of our templates. To best assist you, we advise you to use one of these templates.

I have CC'd a few people who may be able to assist you:

If we have not gotten back to your issue within a few business days, you can try the following:

:owl: Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan.

mattcrdb commented 4 years ago

Hey @benny-wangaimin,

Thanks for opening the GH Issue, we originally discussed the request on https://forum.cockroachlabs.com/t/how-to-ignore-error-rows-during-import/3930

cc @mwang1026