apache / camel-kafka-connector

Camel Kafka Connector allows you to use all Camel components as Kafka Connect connectors
https://camel.apache.org
Apache License 2.0
152 stars 100 forks source link

Possible to use Kafka keys and headers in db inserts? #1653

Open reidmeyer opened 4 months ago

reidmeyer commented 4 months ago

All of the examples for db sinks are like:

INSERT INTO accounts (username,city) VALUES (:#username,:#city)

where the username and city are key names in the json message from the kafka.

Is it possible to insert the kafka key or a kafka header (which is also json)?

davsclaus commented 3 months ago

You can use camel simple language something ala :#${header.kafka.KEY}

reidmeyer commented 3 months ago

HI @davsclaus,

Perhaps I'm missing something super obvious, but I'm struggling to get it to work.

I have a kafka message that:

has value:

{
    "message": "hello there reid"
}

has kafka headers:

{
    "myheader": "myheadervalue"
}

and has a key:

"some-key"

and I have a sql insert like:

camel.kamelet.postgresql-sink.query: 'INSERT INTO mytable (message,other) VALUES (:#message,:#${header.kafka.KEY})'

this doesn't inject anything into header.kafka.KEY, weirdly.

I'm trying to do something like:

camel.kamelet.postgresql-sink.query: 'INSERT INTO mytable (message,mykey,myheader) VALUES (:#message,:#${kafka.key},:#${kafka.header.myheader})', but no luck yet.

Do you know the solution here?