apache / doris

Apache Doris is an easy-to-use, high performance and unified analytics database.
https://doris.apache.org
Apache License 2.0
12.56k stars 3.25k forks source link

doriswriter csv换行,json字段错位 #12142

Open zhoulong0517 opened 2 years ago

zhoulong0517 commented 2 years ago

Search before asking

Version

doris1.1.1

What's Wrong?

同步odps数据到doris,writer:doriswriter,一开始同步的时候format用默认的csv,直接报错,odps表里的字段有\n换行符,把format改成json之后,字段错位

What You Expected?

寻找解决方法,刚刚是不是需要调整分隔符,或者format,谢谢

How to Reproduce?

No response

Anything Else?

No response

Are you willing to submit PR?

Code of Conduct

zhoulong0517 commented 2 years ago

同步odps数据到doris,writer:doriswriter,一开始同步的时候format用默认的csv,直接报错,odps表里的字段有\n换行符,把format改成json之后,字段错位 最开始用 stream load curl命令去导,csv遇到表字段里面有数据是换行的就换行了,导入失败

JNSimba commented 2 years ago

可以考虑使用最新的doris writer,默认是隐藏字符作为分隔符