search

IBM / sarama

Sarama is a Go library for Apache Kafka.
MIT License
11.52k stars 1.75k forks source link

kafka server: The producer attempted to update a transaction while another concurrent operation on the same transaction was ongoing #2915

Open bentcoder opened 5 months ago

bentcoder commented 5 months ago
Description

Using a simple HTTP API where I send many requests in a loop. I create a new producer for each HTTP request using a new UUID just for the sake of testing purposes. Expecting that each producer would work because they all have unique TXN id. But it fails with data race and errors.

I am posting the full example so you can try it yourself. Also look at the logs please.

Versions
Sarama Kafka Go
v1.43.2 v3.3.2 1.22
Configuration

Run it with $ go run -race main.go.

package main

import (
    "log"
    "net/http"
    "time"

    "github.com/IBM/sarama"
    "github.com/google/uuid"
)

var producer = func(id string) (sarama.SyncProducer, error) {
    cfg := sarama.NewConfig()
    cfg.ClientID = "client"
    cfg.Version = sarama.V3_3_2_0
    cfg.Net.MaxOpenRequests = 1
    cfg.Metadata.AllowAutoTopicCreation = false
    cfg.Producer.Return.Successes = true
    cfg.Producer.Compression = sarama.CompressionZSTD
    cfg.Producer.Idempotent = true
    cfg.Producer.RequiredAcks = sarama.WaitForAll
    cfg.Producer.Partitioner = sarama.NewRoundRobinPartitioner
    cfg.Producer.Transaction.ID = "txn-"+id
    cfg.Producer.Transaction.Retry.Max = 10
    cfg.Producer.Transaction.Retry.Backoff = time.Millisecond * 10

    return sarama.NewSyncProducer([]string{":9093"}, cfg)
}

func main() {
    api := api{}

    rtr := http.NewServeMux()
    rtr.HandleFunc("GET /", api.handle)

    log.Println(http.ListenAndServe(":8888", rtr))
}

type api struct{}

func (a api) handle(w http.ResponseWriter, r *http.Request) {
    pro, err := producer(uuid.NewString())
    if err != nil {
        log.Fatalln("FATAL:", err)
    }
    defer pro.Close()

    var (
        batch = 1
        limit = 10
        msgs  = make([]*sarama.ProducerMessage, 0, limit)
    )

    for i := 1; i <= 90; i++ {
        msgs = append(msgs, &sarama.ProducerMessage{
            Topic: "warnings-topic",
            Value: sarama.StringEncoder("test"),
        })

        if len(msgs) == limit {
            if err := pro.BeginTxn(); err != nil {
                log.Println("BeginTxn:", err)
                continue
            }

            if err := pro.SendMessages(msgs); err != nil {
                log.Println("SendMessages:", err)
                continue
            }

            if err := pro.CommitTxn(); err != nil {
                log.Println("CommitTxn:", err)
                continue
            }

            batch++
            msgs = msgs[:0]
        }
    }

    if len(msgs) == limit {
        if err := pro.BeginTxn(); err != nil {
            log.Fatalln("BeginTxn:", err)
        }

        if err := pro.SendMessages(msgs); err != nil {
            log.Fatalln("SendMessages:", err)
        }

        if err := pro.CommitTxn(); err != nil {
            log.Fatalln("CommitTxn:", err)
        }
    }
}
Logs
logs: CLICK ME

``` $ go run -race main.go ================== WARNING: DATA RACE Write at 0x00c000d74910 by goroutine 2731: github.com/IBM/sarama.(*transactionManager).transitionTo() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/transaction_manager.go:220 +0x330 github.com/IBM/sarama.(*asyncProducer).maybeTransitionToErrorState() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/async_producer.go:1281 +0x1c4 github.com/IBM/sarama.(*asyncProducer).returnError() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/async_producer.go:1286 +0xad github.com/IBM/sarama.(*asyncProducer).retryMessage() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/async_producer.go:1323 +0x109 github.com/IBM/sarama.(*asyncProducer).retryMessages() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/async_producer.go:1332 +0x8c github.com/IBM/sarama.(*brokerProducer).handleError.func2() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/async_producer.go:1197 +0x8f4 github.com/IBM/sarama.(*produceSet).eachPartition() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/produce_set.go:223 +0x89b github.com/IBM/sarama.(*brokerProducer).handleError() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/async_producer.go:1196 +0x3cd github.com/IBM/sarama.(*brokerProducer).handleResponse() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/async_producer.go:1060 +0x84 github.com/IBM/sarama.(*brokerProducer).run() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/async_producer.go:993 +0x1a93 github.com/IBM/sarama.(*brokerProducer).run-fm() :1 +0x33 github.com/IBM/sarama.withRecover() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/utils.go:43 +0x41 github.com/IBM/sarama.(*asyncProducer).newBrokerProducer.gowrap1() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/async_producer.go:791 +0x33 Previous read at 0x00c000d74910 by goroutine 2732: github.com/IBM/sarama.(*transactionManager).publishTxnPartitions() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/transaction_manager.go:766 +0x40e github.com/IBM/sarama.(*asyncProducer).newBrokerProducer.func1() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/async_producer.go:818 +0x23c github.com/IBM/sarama.withRecover() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/utils.go:43 +0x41 github.com/IBM/sarama.(*asyncProducer).newBrokerProducer.gowrap2() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/async_producer.go:794 +0x33 Goroutine 2731 (running) created at: github.com/IBM/sarama.(*asyncProducer).newBrokerProducer() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/async_producer.go:791 +0x4f3 github.com/IBM/sarama.(*asyncProducer).getBrokerProducer() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/async_producer.go:1343 +0x133 github.com/IBM/sarama.(*partitionProducer).updateLeaderIfBrokerProducerIsNil.(*partitionProducer).updateLeader.func1() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/async_producer.go:765 +0x244 github.com/eapache/go-resiliency/breaker.(*Breaker).doWork.func1() /Users/me/go/pkg/mod/github.com/eapache/go-resiliency@v1.6.0/breaker/breaker.go:94 +0x83 github.com/eapache/go-resiliency/breaker.(*Breaker).doWork() /Users/me/go/pkg/mod/github.com/eapache/go-resiliency@v1.6.0/breaker/breaker.go:95 +0x3e github.com/eapache/go-resiliency/breaker.(*Breaker).Run() /Users/me/go/pkg/mod/github.com/eapache/go-resiliency@v1.6.0/breaker/breaker.go:58 +0x88 github.com/IBM/sarama.(*partitionProducer).updateLeader() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/async_producer.go:756 +0xca github.com/IBM/sarama.(*partitionProducer).updateLeaderIfBrokerProducerIsNil() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/async_producer.go:619 +0xcb github.com/IBM/sarama.(*partitionProducer).dispatch() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/async_producer.go:689 +0xd08 github.com/IBM/sarama.(*partitionProducer).dispatch-fm() :1 +0x33 github.com/IBM/sarama.withRecover() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/utils.go:43 +0x41 github.com/IBM/sarama.(*asyncProducer).newPartitionProducer.gowrap1() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/async_producer.go:600 +0x33 Goroutine 2732 (running) created at: github.com/IBM/sarama.(*asyncProducer).newBrokerProducer() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/async_producer.go:794 +0x690 github.com/IBM/sarama.(*asyncProducer).getBrokerProducer() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/async_producer.go:1343 +0x133 github.com/IBM/sarama.(*partitionProducer).updateLeaderIfBrokerProducerIsNil.(*partitionProducer).updateLeader.func1() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/async_producer.go:765 +0x244 github.com/eapache/go-resiliency/breaker.(*Breaker).doWork.func1() /Users/me/go/pkg/mod/github.com/eapache/go-resiliency@v1.6.0/breaker/breaker.go:94 +0x83 github.com/eapache/go-resiliency/breaker.(*Breaker).doWork() /Users/me/go/pkg/mod/github.com/eapache/go-resiliency@v1.6.0/breaker/breaker.go:95 +0x3e github.com/eapache/go-resiliency/breaker.(*Breaker).Run() /Users/me/go/pkg/mod/github.com/eapache/go-resiliency@v1.6.0/breaker/breaker.go:58 +0x88 github.com/IBM/sarama.(*partitionProducer).updateLeader() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/async_producer.go:756 +0xca github.com/IBM/sarama.(*partitionProducer).updateLeaderIfBrokerProducerIsNil() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/async_producer.go:619 +0xcb github.com/IBM/sarama.(*partitionProducer).dispatch() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/async_producer.go:689 +0xd08 github.com/IBM/sarama.(*partitionProducer).dispatch-fm() :1 +0x33 github.com/IBM/sarama.withRecover() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/utils.go:43 +0x41 github.com/IBM/sarama.(*asyncProducer).newPartitionProducer.gowrap1() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/async_producer.go:600 +0x33 ================== 2024/05/25 14:14:18 SendMessages: kafka: Failed to deliver 10 messages. 2024/05/25 14:14:18 SendMessages: kafka: Failed to deliver 10 messages. 2024/05/25 14:14:18 SendMessages: kafka: Failed to deliver 8 messages. 2024/05/25 14:14:18 SendMessages: kafka: Failed to deliver 10 messages. 2024/05/25 14:14:18 SendMessages: kafka: Failed to deliver 10 messages. ================== WARNING: DATA RACE Write at 0x00c000ef0550 by goroutine 2858: github.com/IBM/sarama.(*transactionManager).transitionTo() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/transaction_manager.go:220 +0x330 github.com/IBM/sarama.(*asyncProducer).maybeTransitionToErrorState() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/async_producer.go:1281 +0x1c4 github.com/IBM/sarama.(*asyncProducer).returnError() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/async_producer.go:1286 +0xad github.com/IBM/sarama.(*brokerProducer).run() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/async_producer.go:978 +0x1577 github.com/IBM/sarama.(*brokerProducer).run-fm() :1 +0x33 github.com/IBM/sarama.withRecover() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/utils.go:43 +0x41 github.com/IBM/sarama.(*asyncProducer).newBrokerProducer.gowrap1() /Users/me/go/pkg/mod/github.com/!i!b!m/sarama@v1.43.2/async_producer.go:791 +0x33 ================== 2024/05/25 14:14:19 SendMessages: kafka: Failed to deliver 10 messages. 2024/05/25 14:14:25 SendMessages: kafka: Failed to deliver 8 messages. 2024/05/25 14:14:26 SendMessages: kafka: Failed to deliver 8 messages ``` ``` kafka-brooker | [2024-05-25 13:14:14,096] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-60c8ffca-8e22-47d6-8c1b-caeed2f2dd95 with producerId 1651 and producer epoch 0 on partition __transaction_state-24 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,154] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-544f7083-1aa2-4f67-9f88-9475ce955861 with producerId 1652 and producer epoch 0 on partition __transaction_state-48 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,162] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-b63e47ad-fa64-4f39-9e28-cf3740bc2630 with producerId 1656 and producer epoch 0 on partition __transaction_state-39 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,164] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-aa1c4602-d4e1-4cb7-b9ed-f4a11036455f with producerId 1655 and producer epoch 0 on partition __transaction_state-13 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,165] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-03484ebb-8e7c-4e2c-b827-b804263127e6 with producerId 1657 and producer epoch 0 on partition __transaction_state-49 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,165] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-b64ee68d-2448-4a41-a36f-757496caf2bd with producerId 1653 and producer epoch 0 on partition __transaction_state-37 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,179] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-1eb53826-648b-42e6-bc47-e90c39de4742 with producerId 1659 and producer epoch 0 on partition __transaction_state-11 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,182] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-def553c8-663f-4df0-b67e-749fc8d407d6 with producerId 1654 and producer epoch 0 on partition __transaction_state-47 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,184] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-6cee19cf-a555-4c21-be4e-279324ee9775 with producerId 1658 and producer epoch 0 on partition __transaction_state-22 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,189] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-67cd1b3c-f6f6-4a82-b785-b3317ccbc825 with producerId 1660 and producer epoch 0 on partition __transaction_state-35 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,214] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-8fa0bcd7-05f0-4161-a8c3-73f3e7c40c1d with producerId 1661 and producer epoch 0 on partition __transaction_state-14 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,217] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-15659902-8315-4c0d-a68f-c2efbe003588 with producerId 1662 and producer epoch 0 on partition __transaction_state-10 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,220] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-0d77ce2c-deaf-478f-b6b3-b0ed72e510c1 with producerId 1663 and producer epoch 0 on partition __transaction_state-17 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,248] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-05f8e130-07d6-4459-9311-77ff5a60b22e with producerId 1664 and producer epoch 0 on partition __transaction_state-7 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,248] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-3c56b75f-f6c4-4519-a58b-401219b828df with producerId 1666 and producer epoch 0 on partition __transaction_state-21 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,252] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-8ed0948d-ed16-492a-b6f4-6927eac616e3 with producerId 1667 and producer epoch 0 on partition __transaction_state-1 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,257] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-16110e6f-5a29-46fe-b8b2-3f9ca9866f20 with producerId 1668 and producer epoch 0 on partition __transaction_state-10 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,257] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-d0e1a78b-afe7-4c2e-8db6-aa7034dca24e with producerId 1669 and producer epoch 0 on partition __transaction_state-3 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,262] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-85343130-58d2-4270-8155-41a580b9c92f with producerId 1672 and producer epoch 0 on partition __transaction_state-3 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,267] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-e4d09f8b-c164-4caf-a502-9bbdaf4403bb with producerId 1671 and producer epoch 0 on partition __transaction_state-18 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,271] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-713893ad-714d-4ef9-81a0-ed32af82e0d8 with producerId 1670 and producer epoch 0 on partition __transaction_state-25 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,275] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-e5585820-c3da-447b-9784-483da26a8ad8 with producerId 1673 and producer epoch 0 on partition __transaction_state-19 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,293] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-5a600849-13b3-4bcc-8e80-16b418d60e7b with producerId 1674 and producer epoch 0 on partition __transaction_state-28 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,294] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-cf4015c9-92f7-4b50-9bf9-9992371fbffe with producerId 1676 and producer epoch 0 on partition __transaction_state-34 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,298] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-7e646aab-5c44-4e6c-99c2-0257bb1b6358 with producerId 1665 and producer epoch 0 on partition __transaction_state-29 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,293] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-91639ac5-86bc-4ee9-a6a2-478d44e92e9a with producerId 1675 and producer epoch 0 on partition __transaction_state-40 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,300] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-add25b05-01fe-4855-ba47-9766096f3efb with producerId 1677 and producer epoch 0 on partition __transaction_state-13 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,314] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-e908a842-e4d2-465f-af46-d442561d1f91 with producerId 1678 and producer epoch 0 on partition __transaction_state-48 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,318] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-7d1e0300-8478-48e7-a716-42e9e572bdcc with producerId 1679 and producer epoch 0 on partition __transaction_state-40 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,320] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-1e205668-77d2-4614-bb47-d0b5fee5ead6 with producerId 1681 and producer epoch 0 on partition __transaction_state-26 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,321] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-f29c92c1-7aa3-4b80-b7bd-793ec0dd6085 with producerId 1683 and producer epoch 0 on partition __transaction_state-5 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,322] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-0b9ab4fa-771b-4faa-ba6e-d39466b3e555 with producerId 1684 and producer epoch 0 on partition __transaction_state-2 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,324] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-3d9b5563-3605-4f89-85f8-1d85b60ff141 with producerId 1680 and producer epoch 0 on partition __transaction_state-0 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,325] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-b9eb6a5f-2c47-43f1-8679-c72a62100f92 with producerId 1685 and producer epoch 0 on partition __transaction_state-8 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,320] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-06bead21-ce7b-4776-8e3f-032618debe4e with producerId 1682 and producer epoch 0 on partition __transaction_state-12 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,345] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-95970e98-4842-40a1-9aa8-d61501a1f14d with producerId 1686 and producer epoch 0 on partition __transaction_state-48 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,352] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-f26bddea-32a1-45a4-afb0-75b55edaeb9b with producerId 1688 and producer epoch 0 on partition __transaction_state-13 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,356] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-c4e53ab2-4960-4ba5-a612-5037d39748a5 with producerId 1689 and producer epoch 0 on partition __transaction_state-47 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,356] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-1c79bf45-8149-4b9e-9935-9917571bdeae with producerId 1691 and producer epoch 0 on partition __transaction_state-34 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,358] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-641d87bb-e956-4e26-a875-8478ea56edcd with producerId 1687 and producer epoch 0 on partition __transaction_state-40 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,360] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-20eceef5-4264-468d-950e-793756561d1e with producerId 1692 and producer epoch 0 on partition __transaction_state-18 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,362] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-d59f8b77-276d-432a-ae58-82e0a2e6af29 with producerId 1693 and producer epoch 0 on partition __transaction_state-0 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,361] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-6e611d9c-af9f-4149-ab7b-4bb243e5d48d with producerId 1690 and producer epoch 0 on partition __transaction_state-3 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,368] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-fb55be76-7fe3-4f8f-a4dd-3d011251177d with producerId 1694 and producer epoch 0 on partition __transaction_state-47 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,442] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-eaa31c3e-c8e0-4225-a4dc-cb5f393bc8e5 with producerId 1695 and producer epoch 0 on partition __transaction_state-40 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,443] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-8c04c4cd-2b11-4443-b1e6-5fdf42d2ccc2 with producerId 1697 and producer epoch 0 on partition __transaction_state-11 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,444] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-e5430d28-c8d4-476d-a494-60f44d65e418 with producerId 1699 and producer epoch 0 on partition __transaction_state-31 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,444] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-4e60cda6-fbf9-4c40-ad78-9d35cd4bc8ec with producerId 1698 and producer epoch 0 on partition __transaction_state-5 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,447] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-0fa1bc9c-223c-4688-b87d-41e556824384 with producerId 1701 and producer epoch 0 on partition __transaction_state-40 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,451] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-769a47ae-08c2-42f6-a910-6b8937b5f37a with producerId 1696 and producer epoch 0 on partition __transaction_state-2 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,452] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-27d7e4bd-2dcb-426d-a026-4c4f96db5c09 with producerId 1703 and producer epoch 0 on partition __transaction_state-21 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,449] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-f6f7501f-bfec-439a-8cf2-855a128b260b with producerId 1700 and producer epoch 0 on partition __transaction_state-46 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,448] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-24aeff2f-aeab-4bce-b1eb-8ee8ecfcf008 with producerId 1702 and producer epoch 0 on partition __transaction_state-7 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,452] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-70c45aa5-34c6-4574-8def-08d903eaa705 with producerId 1705 and producer epoch 0 on partition __transaction_state-15 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,454] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-93dc10bf-416c-4318-837e-d15fcb917adf with producerId 1706 and producer epoch 0 on partition __transaction_state-9 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,455] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-319d4bad-cf0f-48aa-affc-a160ce8ef980 with producerId 1704 and producer epoch 0 on partition __transaction_state-22 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,472] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-c27c1d46-05a3-4bc5-981d-559501d81efd with producerId 1707 and producer epoch 0 on partition __transaction_state-17 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,474] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-35cfad44-9a4a-48f0-9bea-8eadeb43efb4 with producerId 1708 and producer epoch 0 on partition __transaction_state-32 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,477] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-7b9864f3-dc3f-4942-afaf-22e03771c941 with producerId 1709 and producer epoch 0 on partition __transaction_state-43 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,489] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-fcf4b2b1-fa47-4adb-bf60-78581c3745c7 with producerId 1710 and producer epoch 0 on partition __transaction_state-39 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,498] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-16f11683-8371-47c4-adf5-2d6270070485 with producerId 1711 and producer epoch 0 on partition __transaction_state-21 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,501] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-e3a0e07c-7390-4e5f-97ad-ea0162a2381f with producerId 1712 and producer epoch 0 on partition __transaction_state-33 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,502] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-4e6df14e-f631-4ddd-ab0e-2fcd261437a6 with producerId 1713 and producer epoch 0 on partition __transaction_state-21 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,505] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-feec1d91-20c4-49fd-bdf7-3744041a3159 with producerId 1715 and producer epoch 0 on partition __transaction_state-4 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,506] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-5ff52b0c-cfa6-4113-81bb-47e504f736f2 with producerId 1717 and producer epoch 0 on partition __transaction_state-39 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,506] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-043a0498-fd57-45f5-b3fb-044424b3421d with producerId 1716 and producer epoch 0 on partition __transaction_state-17 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,511] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-1a2448c3-2481-4c39-bf86-e8f3a978c48a with producerId 1718 and producer epoch 0 on partition __transaction_state-44 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,511] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-7a78036b-4de7-4be2-b7b7-9ee6f0ce04c8 with producerId 1719 and producer epoch 0 on partition __transaction_state-38 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,512] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-ecc61022-6904-4c4e-a990-a856daa58bbe with producerId 1721 and producer epoch 0 on partition __transaction_state-13 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,512] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-25225f4b-0292-48d2-b59a-736cd4892260 with producerId 1720 and producer epoch 0 on partition __transaction_state-17 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,512] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-cf1b920c-b1ca-4f92-99fb-3d9dc0b2b8c0 with producerId 1722 and producer epoch 0 on partition __transaction_state-7 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,513] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-85542dc2-e564-4972-8d4d-c67e572aea73 with producerId 1723 and producer epoch 0 on partition __transaction_state-30 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,513] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-66b951a4-42d9-43ab-b90e-b22a36bfc882 with producerId 1714 and producer epoch 0 on partition __transaction_state-9 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,514] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-721b4382-7667-4283-977c-73e95b8d03eb with producerId 1724 and producer epoch 0 on partition __transaction_state-38 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,517] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-b522bc3d-c81d-4c8e-849a-e3cfbc1f408b with producerId 1725 and producer epoch 0 on partition __transaction_state-7 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,523] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-7fb066f2-e342-455d-a16a-59425bd2d8e1 with producerId 1726 and producer epoch 0 on partition __transaction_state-12 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,554] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-007e9202-2d32-4580-ab22-8ff22c8dd492 with producerId 1727 and producer epoch 0 on partition __transaction_state-29 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,556] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-02954028-6364-499d-abb2-9beea9f7ccf7 with producerId 1728 and producer epoch 0 on partition __transaction_state-46 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,559] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-d69c13da-3e1a-4eee-8c5e-912829c58779 with producerId 1729 and producer epoch 0 on partition __transaction_state-6 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,561] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-5be890f0-7f05-47b5-b7df-b5b6a39b97a9 with producerId 1731 and producer epoch 0 on partition __transaction_state-7 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,560] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-296b3b67-733d-4985-83ef-99bd775eddae with producerId 1730 and producer epoch 0 on partition __transaction_state-31 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,627] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-02b28be0-d5eb-4722-80cd-b7652336733f with producerId 1732 and producer epoch 0 on partition __transaction_state-23 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,627] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-b8b0533c-09c2-4197-afa5-feb4d4e93fb2 with producerId 1733 and producer epoch 0 on partition __transaction_state-39 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,648] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-5bac5251-de15-4593-b5bf-e843b3c90fa5 with producerId 1734 and producer epoch 0 on partition __transaction_state-29 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,648] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-681d3362-ceb7-4ea9-a33a-7aff03f48ffb with producerId 1735 and producer epoch 0 on partition __transaction_state-7 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,739] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-19416ac4-8de5-4830-bf4a-5fed04865333 with producerId 1736 and producer epoch 0 on partition __transaction_state-32 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,777] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-77b8ad69-4af3-47c1-bb66-a49cdd336286 with producerId 1737 and producer epoch 0 on partition __transaction_state-5 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,791] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-4c4ce6a8-b68c-4273-b512-5ddf8f3d9161 with producerId 1738 and producer epoch 0 on partition __transaction_state-43 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,792] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-d64848ad-bf46-4591-9d73-8d5364b46009 with producerId 1739 and producer epoch 0 on partition __transaction_state-27 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,795] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-42fa3444-8cc3-4f35-acd6-cbe0ba1b4647 with producerId 1741 and producer epoch 0 on partition __transaction_state-11 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,796] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-f3c001a7-e1da-4451-b76a-82ac032b39fa with producerId 1742 and producer epoch 0 on partition __transaction_state-43 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,817] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-f4132ad3-3284-49a1-aa6e-8f4eea24eeaa with producerId 1745 and producer epoch 0 on partition __transaction_state-43 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,816] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-b584baca-2900-4845-b0b9-48a5e9182077 with producerId 1744 and producer epoch 0 on partition __transaction_state-49 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,801] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-ac39512c-067d-4f30-ab65-ba977174216f with producerId 1743 and producer epoch 0 on partition __transaction_state-5 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,799] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-2f0bc642-417c-4fcc-8c35-fd910abff460 with producerId 1740 and producer epoch 0 on partition __transaction_state-5 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,827] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-77bf24d3-b1c5-433f-82a5-a60ba78338c1 with producerId 1746 and producer epoch 0 on partition __transaction_state-34 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,827] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-34d68923-502a-46c5-9c8d-170701cf8b86 with producerId 1748 and producer epoch 0 on partition __transaction_state-7 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,829] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-f9c680ec-e394-4bbe-a64e-74d9e3a16030 with producerId 1749 and producer epoch 0 on partition __transaction_state-14 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:14,832] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-973b5237-83bc-45cd-ac97-cabd6fed885b with producerId 1747 and producer epoch 0 on partition __transaction_state-9 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:17,761] ERROR [ReplicaManager broker=1] Error processing append operation on partition warnings-topic-0 (kafka.server.ReplicaManager) kafka-brooker | org.apache.kafka.common.errors.OutOfOrderSequenceException: Out of order sequence number for producer 1674 at offset 548171 in partition warnings-topic-0: 51 (incoming seq. number), 49 (current end sequence number) kafka-brooker | [2024-05-25 13:14:17,891] ERROR [ReplicaManager broker=1] Error processing append operation on partition warnings-topic-0 (kafka.server.ReplicaManager) kafka-brooker | org.apache.kafka.common.errors.OutOfOrderSequenceException: Out of order sequence number for producer 1658 at offset 548305 in partition warnings-topic-0: 51 (incoming seq. number), 49 (current end sequence number) kafka-brooker | [2024-05-25 13:14:17,962] ERROR [ReplicaManager broker=1] Error processing append operation on partition warnings-topic-0 (kafka.server.ReplicaManager) kafka-brooker | org.apache.kafka.common.errors.OutOfOrderSequenceException: Out of order sequence number for producer 1659 at offset 548366 in partition warnings-topic-0: 52 (incoming seq. number), 49 (current end sequence number) kafka-brooker | [2024-05-25 13:14:18,167] ERROR [ReplicaManager broker=1] Error processing append operation on partition warnings-topic-0 (kafka.server.ReplicaManager) kafka-brooker | org.apache.kafka.common.errors.OutOfOrderSequenceException: Out of order sequence number for producer 1675 at offset 548501 in partition warnings-topic-0: 51 (incoming seq. number), 49 (current end sequence number) kafka-brooker | [2024-05-25 13:14:18,242] ERROR [ReplicaManager broker=1] Error processing append operation on partition warnings-topic-0 (kafka.server.ReplicaManager) kafka-brooker | org.apache.kafka.common.errors.OutOfOrderSequenceException: Out of order sequence number for producer 1677 at offset 548533 in partition warnings-topic-0: 51 (incoming seq. number), 49 (current end sequence number) kafka-brooker | [2024-05-25 13:14:18,553] ERROR [ReplicaManager broker=1] Error processing append operation on partition warnings-topic-0 (kafka.server.ReplicaManager) kafka-brooker | org.apache.kafka.common.errors.OutOfOrderSequenceException: Out of order sequence number for producer 1676 at offset 549024 in partition warnings-topic-0: 53 (incoming seq. number), 51 (current end sequence number) kafka-brooker | [2024-05-25 13:14:18,702] ERROR [ReplicaManager broker=1] Error processing append operation on partition warnings-topic-0 (kafka.server.ReplicaManager) kafka-brooker | org.apache.kafka.common.errors.OutOfOrderSequenceException: Out of order sequence number for producer 1676 at offset 549260 in partition warnings-topic-0: 51 (incoming seq. number), 51 (current end sequence number) kafka-brooker | [2024-05-25 13:14:19,147] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-36bfbd1f-2a2e-4537-9ffa-b727b7ce3ca9 with producerId 1750 and producer epoch 0 on partition __transaction_state-39 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:19,161] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-ef1bcf44-d7e3-4d0e-ba73-c4afb9397acc with producerId 1751 and producer epoch 0 on partition __transaction_state-18 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:19,212] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-5cdc56f8-b363-4231-ba26-c22d7a9e04a2 with producerId 1752 and producer epoch 0 on partition __transaction_state-16 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:19,225] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-a9def587-21ce-4943-989e-c0f9a9b62662 with producerId 1753 and producer epoch 0 on partition __transaction_state-20 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:19,344] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-0057b872-6278-4cec-93c8-88608980bdfc with producerId 1754 and producer epoch 0 on partition __transaction_state-5 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:19,757] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-005193e8-883b-4b22-8944-98631742c4b0 with producerId 1755 and producer epoch 0 on partition __transaction_state-11 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:20,287] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-38afc751-305a-44a3-a988-189470668cae with producerId 1756 and producer epoch 0 on partition __transaction_state-14 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:20,307] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-96490e7a-a60c-431b-837e-541957366496 with producerId 1757 and producer epoch 0 on partition __transaction_state-2 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:20,534] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-75f45520-d524-4ef6-aab4-cf8fdec3e761 with producerId 1758 and producer epoch 0 on partition __transaction_state-24 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:20,618] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-40df57f1-ca79-4936-a2a9-3f9310753332 with producerId 1759 and producer epoch 0 on partition __transaction_state-10 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:20,667] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-c773a90c-d290-407e-96f6-27d975035dca with producerId 1760 and producer epoch 0 on partition __transaction_state-30 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:20,701] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-b1318663-620e-40fd-b0e4-4db1410ccecb with producerId 1761 and producer epoch 0 on partition __transaction_state-11 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:20,718] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-1e6acd58-efb6-413e-b69a-d4a82842b4a5 with producerId 1762 and producer epoch 0 on partition __transaction_state-37 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:20,778] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-abbc2137-5b15-443d-a231-d43c48c79f1f with producerId 1763 and producer epoch 0 on partition __transaction_state-5 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:20,872] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-22a59dab-c6a9-4299-81b9-f0e8c41cbd93 with producerId 1764 and producer epoch 0 on partition __transaction_state-16 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:20,944] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-59bae99b-e2c8-4747-b375-ceb1f393cd08 with producerId 1765 and producer epoch 0 on partition __transaction_state-29 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:20,999] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-f6224f81-661c-46e4-9ee2-a03ce417e385 with producerId 1766 and producer epoch 0 on partition __transaction_state-1 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,033] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-345ce160-3ab5-48b6-bbe2-decd200ac98b with producerId 1767 and producer epoch 0 on partition __transaction_state-3 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,033] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-2355b1c8-d652-473a-8858-9870408220e5 with producerId 1768 and producer epoch 0 on partition __transaction_state-29 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,071] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-5805a7b6-47cc-4aa6-b18b-4eb82d077c97 with producerId 1769 and producer epoch 0 on partition __transaction_state-19 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,114] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-0ed0a632-1e2c-4bc8-9550-e7b6038d63f1 with producerId 1770 and producer epoch 0 on partition __transaction_state-18 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,166] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-dea5aa46-c8b8-4243-9e7f-39934fab3974 with producerId 1771 and producer epoch 0 on partition __transaction_state-37 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,168] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-17d335e6-08de-4e48-a25b-6fc424a4e950 with producerId 1772 and producer epoch 0 on partition __transaction_state-0 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,244] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-a954f028-c349-4265-82f0-e20721d7ae3c with producerId 1773 and producer epoch 0 on partition __transaction_state-38 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,272] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-ef34fd85-b4e0-4c2e-af66-e7d0cde52fea with producerId 1774 and producer epoch 0 on partition __transaction_state-9 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,272] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-8c14eb1f-5b5e-4976-94ba-dcd70e34db0b with producerId 1775 and producer epoch 0 on partition __transaction_state-22 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,296] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-84755b98-e49c-4ff8-910f-9aa9e5b5c386 with producerId 1776 and producer epoch 0 on partition __transaction_state-11 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,321] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-b0168931-209e-46cb-99b3-038d31686736 with producerId 1777 and producer epoch 0 on partition __transaction_state-4 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,338] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-c42cdf8b-2b8d-4bcb-a652-4a1cb8976def with producerId 1778 and producer epoch 0 on partition __transaction_state-18 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,343] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-9e0d1ba6-886b-4ac9-9c70-12315bd54bcb with producerId 1780 and producer epoch 0 on partition __transaction_state-33 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,344] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-66211dad-e08b-417f-8ae2-b8c4f88293d7 with producerId 1779 and producer epoch 0 on partition __transaction_state-39 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,357] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-362f4d9c-9b10-4657-951d-743509064868 with producerId 1781 and producer epoch 0 on partition __transaction_state-31 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,358] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-c8f59047-ab62-42b0-9ce5-114741290335 with producerId 1782 and producer epoch 0 on partition __transaction_state-32 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,432] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-fc1101e2-2489-4e1d-8e3a-d1a89676a3a4 with producerId 1784 and producer epoch 0 on partition __transaction_state-13 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,434] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-802cbb6e-aef2-403a-8c1c-e9b794e5da9a with producerId 1785 and producer epoch 0 on partition __transaction_state-43 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,438] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-7e3983b9-f6ac-4a4e-bfaa-71c6903ea08c with producerId 1786 and producer epoch 0 on partition __transaction_state-11 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,441] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-e9cbfdb5-2698-4501-8f06-4be425d5cdfb with producerId 1787 and producer epoch 0 on partition __transaction_state-8 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,444] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-784fde80-00e0-4606-976a-a859cf69e694 with producerId 1788 and producer epoch 0 on partition __transaction_state-24 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,432] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-174d836c-44d2-46fb-b09a-3751ae3750fc with producerId 1783 and producer epoch 0 on partition __transaction_state-41 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,459] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-0a62315d-b94c-4afa-afd4-08bc66d8fd01 with producerId 1789 and producer epoch 0 on partition __transaction_state-40 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,504] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-02be42fc-d017-4faf-91e0-6314c9b50f15 with producerId 1790 and producer epoch 0 on partition __transaction_state-6 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,581] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-ac915372-afc2-4168-8f0a-51368929e472 with producerId 1791 and producer epoch 0 on partition __transaction_state-23 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,581] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-4569969d-cdd8-4b51-a49e-bd5ace2c3366 with producerId 1792 and producer epoch 0 on partition __transaction_state-2 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,597] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-d80675c7-e7ca-485a-b4b4-7f63a1795a18 with producerId 1793 and producer epoch 0 on partition __transaction_state-36 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,614] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-b785f278-911c-45e1-8720-bdd4820fd000 with producerId 1794 and producer epoch 0 on partition __transaction_state-38 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,618] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-ebbd06fc-adfc-45bb-a687-31312f074561 with producerId 1795 and producer epoch 0 on partition __transaction_state-3 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,670] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-ae18d793-4b07-4740-b542-8a78482d3ecf with producerId 1796 and producer epoch 0 on partition __transaction_state-11 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,675] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-57012afb-efc2-4da3-9de0-f1d551bfbf54 with producerId 1797 and producer epoch 0 on partition __transaction_state-2 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,678] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-84023299-98d5-4b59-8c95-234c0776763e with producerId 1798 and producer epoch 0 on partition __transaction_state-24 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,679] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-21d2aedf-eedb-4f71-854f-66b5eaabf158 with producerId 1800 and producer epoch 0 on partition __transaction_state-5 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,680] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-7cf2db76-75b7-44e5-8767-ce4c20826c29 with producerId 1799 and producer epoch 0 on partition __transaction_state-40 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,716] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-3a2f23ce-36c0-497f-af5d-31aa9d482e3c with producerId 1801 and producer epoch 0 on partition __transaction_state-48 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,746] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-a07afa81-1764-4be5-8345-ea7b62072fe9 with producerId 1802 and producer epoch 0 on partition __transaction_state-2 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,749] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-dc170a0d-f382-4f8b-9ab1-5384ab43b432 with producerId 1803 and producer epoch 0 on partition __transaction_state-36 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,749] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-0082dd3b-a48d-4fdb-bfc0-ddd456eeac94 with producerId 1804 and producer epoch 0 on partition __transaction_state-42 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,822] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-88b5f77e-14e0-4ab8-bba3-70ce73b9e7fe with producerId 1805 and producer epoch 0 on partition __transaction_state-27 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,828] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-ffca29da-b258-4712-a848-e5208eb6e75f with producerId 1807 and producer epoch 0 on partition __transaction_state-9 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,823] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-e6129db3-6e9e-49ac-bd1a-9ac097b4ef45 with producerId 1806 and producer epoch 0 on partition __transaction_state-6 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,855] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-faffe101-34e4-4e5e-a039-1cf2a8b20782 with producerId 1808 and producer epoch 0 on partition __transaction_state-29 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,857] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-db3c82d9-631c-4693-add9-0864951ea049 with producerId 1809 and producer epoch 0 on partition __transaction_state-29 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,906] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-34f8064a-503a-48bc-aadb-7f99cf635c91 with producerId 1810 and producer epoch 0 on partition __transaction_state-12 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:21,984] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-b4a20b45-3120-43a7-9176-44fd3cd2e57e with producerId 1811 and producer epoch 0 on partition __transaction_state-33 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:22,099] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-761cb723-4a4c-48a5-9cd8-aa27c55b3c00 with producerId 1812 and producer epoch 0 on partition __transaction_state-14 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:22,272] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-062fe8be-8c7b-44a7-a3bf-c2ac432312f3 with producerId 1813 and producer epoch 0 on partition __transaction_state-12 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:22,335] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-018e6d0e-879c-42fe-9249-619f66962f75 with producerId 1814 and producer epoch 0 on partition __transaction_state-21 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:22,340] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-6d570805-1271-452b-96dc-9615dffbd9d3 with producerId 1815 and producer epoch 0 on partition __transaction_state-21 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:22,342] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-51023310-7960-4d19-b99f-166b1c5a7df6 with producerId 1817 and producer epoch 0 on partition __transaction_state-46 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:22,347] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-532b9e11-2b0c-45e6-b310-399238d5e0fd with producerId 1818 and producer epoch 0 on partition __transaction_state-34 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:22,342] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-6fae9d12-05d1-4313-b9ce-6503c6c8f892 with producerId 1816 and producer epoch 0 on partition __transaction_state-6 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:22,376] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-05b225df-dd2e-4243-9e23-22cd4fa93a7b with producerId 1819 and producer epoch 0 on partition __transaction_state-30 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:22,513] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-02947075-b4f3-406f-9aaf-07b0c766bfa7 with producerId 1820 and producer epoch 0 on partition __transaction_state-30 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:22,656] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-951b93b7-d983-48ae-9eb5-86e4bdcdefe5 with producerId 1821 and producer epoch 0 on partition __transaction_state-37 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:22,662] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-c120947b-7753-48fd-a7bf-c857c6ce474a with producerId 1822 and producer epoch 0 on partition __transaction_state-39 (kafka.coordinator.transaction.TransactionCoordinator) kafka-brooker | [2024-05-25 13:14:22,671] INFO [TransactionCoordinator id=1] Initialized transactionalId txn-cd343c7b-e2f3-4517-bcfc-5ca1eeaa1a77 with producerId 1823 and producer epoch 0 on partition __transaction_state-9 (kafka.coordinator.transaction.TransactionCoordinator) kafka-ui | 2024-05-25 13:14:39,196 DEBUG [parallel-1] c.p.k.u.s.ClustersStatisticsScheduler: Start getting metrics for kafkaCluster: local kafka-ui | 2024-05-25 13:14:39,219 DEBUG [parallel-2] c.p.k.u.s.ClustersStatisticsScheduler: Metrics updated for cluster: local kafka-brooker | [2024-05-25 13:14:47,804] INFO [LocalLog partition=warnings-topic-0, dir=/bitnami/kafka/data] Deleting segment files LogSegment(baseOffset=543303, size=78, lastModifiedTime=1716642717524, largestRecordTimestamp=Some(1716642717524)) (kafka.log.LocalLog$) kafka-brooker | [2024-05-25 13:14:47,806] INFO Deleted log /bitnami/kafka/data/warnings-topic-0/00000000000000543303.log.deleted. (kafka.log.LogSegment) kafka-brooker | [2024-05-25 13:14:47,806] INFO Deleted offset index /bitnami/kafka/data/warnings-topic-0/00000000000000543303.index.deleted. (kafka.log.LogSegment) kafka-brooker | [2024-05-25 13:14:47,807] INFO Deleted time index /bitnami/kafka/data/warnings-topic-0/00000000000000543303.timeindex.deleted. (kafka.log.LogSegment) kafka-brooker | [2024-05-25 13:14:47,835] INFO Deleted producer state snapshot /bitnami/kafka/data/warnings-topic-0/00000000000000543303.snapshot.deleted (kafka.log.SnapshotFile) ```

Additional Context

I use https://k6.io/ (install) to send requests using script below if you want to use it.

import http from "k6/http";
import { check } from "k6";

export let options = {
    vus: 99,
    duration: "10s"
};

export default function() {
    let response = http.get("http://localhost:8888/");

    check(response, {
        "200 OK": (r) => r.status === 200,
        "response < 100 ms": (r) => r.timings.duration < 100
    });
}
bentcoder commented 5 months ago

Relates to:

nendic commented 3 months ago

did you find a solution to the problem?

bentcoder commented 3 months ago

https://www.inanzzz.com/index.php/post/8yvd/creating-a-sync-producer-pool-for-transactional-kafka-events-in-golang

github-actions[bot] commented 1 week ago

Thank you for taking the time to raise this issue. However, it has not had any activity on it in the past 90 days and will be closed in 30 days if no updates occur. Please check if the main branch has already resolved the issue since it was raised. If you believe the issue is still valid and you would like input from the maintainers then please comment to ask for it to be reviewed.