search

bakdata / kafka-error-handling

A library for error handling in Kafka Streams.
MIT License
18 stars 3 forks source link

Proto conversion for deadletter #13

Closed mkcode92 closed 2 years ago

torbsto commented 2 years ago

You have to include the protoc artifact in build file of the proto project, see brute-force-serde:

import com.google.protobuf.gradle.protobuf
import com.google.protobuf.gradle.protoc

...

protobuf {
    protoc {
        artifact = "com.google.protobuf:protoc:3.18.1"
    }
}
torbsto commented 2 years ago

You can increase Gradle's heap space in gradle.properties, e.g. org.gradle.jvmargs=-Xmx2048m

philipp94831 commented 2 years ago

Let's test if the library is usable. Please add ref: tmp/upload-snapshot to the azure template https://github.com/bakdata/kafka-error-handling/blob/master/azure-pipelines.yml#L18. It will then be published to Sonatype Snapshot Repo (usually only happens on master branch)

mkcode92 commented 2 years ago

tested the snapshot manually in one of our projects, works both for avro and proto as expected.

mkcode92 commented 2 years ago

At which version should we start with the new libraries?

philipp94831 commented 2 years ago

At which version should we start with the new libraries?

This will be handled by the release pipeline. We will use 1.3 or 2.0, not sure yet

philipp94831 commented 2 years ago

So I updated the dead letter analyzer but there is a strange issue which I don't quite understand https://dev.azure.com/bakdata/public/_build/results?buildId=20439&view=logs&j=ca395085-040a-526b-2ce8-bdc85f692774&t=85c38824-6449-5223-6cf4-ccf7ebb27e39

philipp94831 commented 2 years ago

So I updated the dead letter analyzer but there is a strange issue which I don't quite understand https://dev.azure.com/bakdata/public/_build/results?buildId=20439&view=logs&j=ca395085-040a-526b-2ce8-bdc85f692774&t=85c38824-6449-5223-6cf4-ccf7ebb27e39

Nevermind, we just have a name clash there. The dead letter analyzer also has an interface DeadLetterConverter in the same package