Serde: Enable header modifications by custom serializer

[MDzajaS]

Issue submitter TODO list

• [X] I've searched for an already existing issues here
• [X] I'm running a supported version of the application which is listed here and the feature is not present there

Is your proposal related to a problem?

Some schema registries allow placing schema details in message headers instead of the payload. For example, Apicurio Registry serializer enables writing the artifact identifier to message headers.

However, when creating custom serializers that implement the io.kafbat.ui.serde.api.Serde interface with io.kafbat.ui.serde.api.Serde.Serializer, these interfaces don't provide direct access to message headers, which can be limiting.

Describe the feature you're interested in

Expend io.kafbat.ui.serde.api.Serde.Serializer to support message headers editing.

The least complicated solution would be to extend the serializer interface with additional method:

default byte[] serialize(String input, Headers headers) {
    return serialize(input);
}

Where Headers is org.apache.kafka.common.header.Headers, a mutable collection of headers.

If immutable headers are required (e.g., io.kafbat.ui.serde.api.RecordHeaders), then the interface might be extended with this method:

default SerializeResult serialize(String input, RecordHeaders headers) {
    return new SerializeResult(serialize(input), headers);
}

Where SerializeResult would be a new class that contains serialized message and headers.

By adding this method, io.kafbat.ui.serde.api.Serde.Serializer would move one step closer to Kafka's org.apache.kafka.common.serialization.Serializer interface.

Describe alternatives you've considered

There is no alternative for this feature. For instance, when using Apicurio Registry, if a serializer includes schema details directly in the message payload, it prepends a magic byte to the beginning of the message. If the deserializer doesn't expect this magic byte, it will not be able to read the message. So, to use Kafka-UI to produce messages with Apicurio Registry, it is necessary to modify message headers in the serializer.

Version you're running

2956664

Additional context

No response

[github-actions[bot]]

Hi MDzajaS! 👋

Welcome, and thank you for opening your first issue in the repo!

Please wait for triaging by our maintainers.

As development is carried out in our spare time, you can support us by sponsoring our activities or even funding the development of specific issues. Sponsorship link

If you plan to raise a PR for this issue, please take a look at our contributing guide.

[MDzaja]

Hi, I'm writing from my private account. I took a look at the code and created a solution. I know that this issue is still not in the "Up for grabs" category, but I would like to be assigned to it once it is. Assign it to this account, and I'll make a PR where we can further discuss implementation details.

Thank you.

[Haarolean]

@MDzaja I remember that we did not support headers on purpose but I don't remember the exact reason for it. Let me discuss it with the team this week and I'll get back to you.

[MDzaja]

@Haarolean Is there any update on this issue.

[MDzaja]

@Haarolean I've created a PR with an initial solution proposal on my personal fork of the repo. Is there any progress on considering this feature?

[Haarolean]

@MDzaja sorry for the delay. We came to a conclusion that we didn't initially extend the interface for headers serialization just because we thought nobody would need that. Feel free to raise a PR, we'll take a look.