search

big-armor / datapm

Other
18 stars 5 forks source link

Timeplus sink: use empty string for null or {}, stringify json, ignore failed ingestion #490

Closed jovezhong closed 2 years ago

jovezhong commented 2 years ago

I found the Timeplus server will report error if one field is {} or null, in this fix, use " " (a whitespace) as the value for complex value("object" type), directly sending the raw object can be risky too, so in this fix, change it to a string via JSON.stringify(columnValue)

Another important enhancement to skip the ingestion if the REST API returns error. Use jobContext.print("WARN" to show the warning and carry on

traviscollins commented 2 years ago

This is include in 0.31.15, and is now up on datapm.io