search

confluentinc / demo-scene

👾Scripts and samples to support Confluent Demos and Talks. ⚠️Might be rough around the edges ;-) 👉For automated tutorials and QA'd code, see https://github.com/confluentinc/examples/
https://developer.confluent.io
Apache License 2.0
1.5k stars 896 forks source link

Flatten nested JSON data fields and whitelist them and Flatten SMT fails on arrays #240

Closed vishaldhanani closed 2 years ago

vishaldhanani commented 2 years ago

Hello,

I am getting the below error while trying to flatten nested JSON data in my elastic sink connector. my main purpose is to whitelist the required fields for my elastic index. When I tried to use a direct whitelist that will not consider nested fields in it. so that's the reason to do flatten first and then rename those nested fields and then whitelist those required fields. Is there any other way we can achieve this?

Caused by: org.apache.kafka.connect.errors.DataException: Flatten transformation does not support class java.util.ArrayList for record without schemas

Thank you for your help in advance!

rmoff commented 2 years ago

Hi @vishaldhanani , please can you post this over at https://forum.confluent.io/ thanks.

vishaldhanani commented 2 years ago

Hi @rmoff, I have already posted in the Confluent connect slack channel but didn't get any response.

Thank you!