When I followed the Kafka-connector-starrocks guide to load data to StarRocks, I got the error message below.
ERROR [starrocks-kafka-connector|task-0] catch exception, wait rollback (com.starrocks.data.load.stream.v2.StreamLoadManagerV2:424)
com.starrocks.data.load.stream.exception.StreamLoadFailException: Transaction prepare failed, db: example_db, table: test-sqlite-jdbc-accounts, label: -0aba9ed9-fd83-41c3-afcf-9be12bbc926f,
responseBody: {
"Status": "DATA_QUALITY_ERROR",
"Message": "Failed to iterate document stream as object. error: The JSON element does not have the requested type.",
"ErrorURL": "http://10.5.0.3:8040/api/_load_error_log?file=error_log_174479fdeb882113_29549abd35d3bdb2"
}
errorLog: Error: Data quality error: Failed to iterate document stream as object. error: The JSON element does not have the requested type.. Row: parser current location: [{"id":19,"name":"cathy"}]
at com.starrocks.data.load.stream.TransactionStreamLoader.prepare(TransactionStreamLoader.java:221)
at com.starrocks.data.load.stream.v2.TransactionTableRegion.commit(TransactionTableRegion.java:247)
at com.starrocks.data.load.stream.v2.StreamLoadManagerV2.lambda$init$0(StreamLoadManagerV2.java:210)
at java.lang.Thread.run(Thread.java:750)
I can get the same error message if I run the command below.
But if I add a header by -H "strip_outer_array:true", it will succeed.
So I fix the error by adding "sink.properties.strip_outer_array=true" to the properties file of Kafka-connector-starrocks plugin, because the configuration item will cause kafka-connector-starrocks adding "strip_outer_array:true" to http request header when do stream load.
My question is, is it possible to add the "strip_outer_array:true" header automatically by kafka-connector-starrocks, or do not wrap the data with '[ ]'?
When I followed the Kafka-connector-starrocks guide to load data to StarRocks, I got the error message below.
I can get the same error message if I run the command below.
But if I add a header by
-H "strip_outer_array:true"
, it will succeed. So I fix the error by adding "sink.properties.strip_outer_array=true" to the properties file of Kafka-connector-starrocks plugin, because the configuration item will cause kafka-connector-starrocks adding "strip_outer_array:true" to http request header when do stream load. My question is, is it possible to add the "strip_outer_array:true" header automatically by kafka-connector-starrocks, or do not wrap the data with '[ ]'?BTW, I submitted a PR to modify the guide by adding "sink.properties.strip_outer_array=true" to the properties file. https://github.com/StarRocks/starrocks/pull/41218