swift-server / swift-kafka-client

Apache License 2.0
81 stars 20 forks source link

A lot of idling in KafkaConsumer.batchConsumerPoll #151

Closed blindspotbounty closed 10 months ago

blindspotbounty commented 10 months ago

After https://github.com/swift-server/swift-kafka-client/pull/139 we noticed that there are a lot of idling in KafkaConsumer.batchConsumerPoll

Seems that now for-loop does not exit when there is no more messages:

    private func batchConsumerPoll(
        client: RDKafkaClient,
        maxMessages: Int = 100
    ) -> [Result<KafkaConsumerMessage, Error>] {
        var messageResults = [Result<KafkaConsumerMessage, Error>]()
        messageResults.reserveCapacity(maxMessages)

        for _ in 0..<maxMessages {
            var result: Result<KafkaConsumerMessage, Error>?
            do {
                if let message = try client.consumerPoll() {
                    result = .success(message)
                }
            } catch {
                result = .failure(error)
            }

            if let result {
                messageResults.append(result)
            }                                          ///   <-------- seems we should else { return messageResults }
        }

        return messageResults
    }

Not sure if it is intended. Could you advise, please?

FranzBusch commented 10 months ago

Yes we should be doing that. Do you mind opening a PR?