Transactional producer is does not commit message causing `Local: Erroneous state`

[Fraktala]

Description

I tried to produce avro messages to Redpanda using transactional approach. After I send all of the data on_delivery provided error: cimpl.KafkaException: KafkaError{code=_STATE,val=-172,str="Unable to produce message: Local: Erroneous state"}

Same error appeared when I did not set on_delivery function but I tried to read from same topic.

In Redpanda UI I cannot see those messages but I can check that Message Count has been set higher. It seems like messages has not been commited.

My issue may be related to this issue.

How to reproduce

Publish data using serialised message with transaction.

>>> confluent_kafka.version()
... ('2.3.0', 33751040)
>>> confluent_kafka.libversion()
... ('2.3.0', 33751295)

Problem occurred at: Redpanda v23.3.4 Auto-commit has been set to: False

Checklist

Please provide the following information:

• [ ] confluent-kafka-python and librdkafka version (confluent_kafka.version() and confluent_kafka.libversion()):
• [ ] Apache Kafka broker version:
• [ ] Client configuration: {...}
• [ ] Operating system:
• [ ] Provide client logs (with 'debug': '..' as necessary)
• [ ] Provide broker log excerpts
• [ ] Critical issue

[Fraktala]

For unknown reason when I tried to reproduce error on smaller example, it stopped to showing off - on production too. So right now I cannot be sure was a code an issue.