newrelic / kafka-connect-newrelic

Apache License 2.0
5 stars 17 forks source link

Connector should have a convertor for the Metrics JSON payload #15

Closed thtmnisamnstr closed 3 years ago

thtmnisamnstr commented 3 years ago

Description

This connector should have a convertor for the Metrics JSON payload. It currently uses a standard JSON converter, which works but has downsides. Rahul (from Confluent) built a converter for Event JSON payload. This needs the same for Metrics.

Previous Correspondence FROM: Rahul Bhattacharya, Sep 7, 2020, 9:05 AM (10 days ago)

Hi All, we had a video call with the Hashedin team and figured out the issue. Now they are unblocked. I had a couple of questions from the NewRelic end. I had developed a Convertor for the events json, but i could not do it for the Metrics payload as i was out of time. Are we going with the approach of using an event Converter for Events and JsonConverter for Metrics. We can do that, but as we discussed the benefit of using the Converter will be any deserialization errors can be redirected to DLQs which won't be available with a JSON converter.

regards


FROM: Stuart MacKenzie, Sep 8, 2020, 6:21 AM

Rahul,

Those are both examples that will be published to help the customer author any transforms they may need. In my mind, it shows / demonstrates how to use the Matrics API & how to use the Events API. After release, we'll continue to add additional converters for "out of the box" capabilities.

Let me know if that answers your question.

- Stuart