confluentinc / kafka-connect-storage-common

Shared software among connectors that target distributed filesystems and cloud storage.
Other
5 stars 155 forks source link

Use record.newRecord instead of Creating new SinkRecord #347

Closed naveenmall11 closed 7 months ago

naveenmall11 commented 7 months ago

Problem

https://confluent.slack.com/archives/C06HJ2VFQ5T While projecting we were changing instance of record from InternalSinkRecord to SinkRecord and while reporting we were using modified records which was causing the original schema of messages to be changed while serialising again.

Solution

Instead of creating a new object, we can use record.newRecord. Functionality wise it will same for connector and while reporting we will be able to report instance of `InternalSinkRecord'

Does this solution apply anywhere else?
If yes, where?

Test Strategy

Tested the change with reproduction model https://github.com/confluentinc/kafka-docker-playground-internal/blob/5fbb61804e4a282ccd29cc77719bfc8cc14d19ed/connect-connect-aws-s3-sink/s3-sink-repro-194748-unexpected-register-of-schema.sh

After the fix, no unwanted version of schema register was found.

image image
Testing done:

Release Plan

cla-assistant[bot] commented 7 months ago

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

cla-assistant[bot] commented 7 months ago

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.