search

fd4s / fs2-kafka

Functional Kafka Streams for Scala
https://fd4s.github.io/fs2-kafka
Apache License 2.0
293 stars 101 forks source link

AdminApiClient deleteTopic doesn't wait for completion. #744

Closed paul-snively closed 8 months ago

paul-snively commented 3 years ago

In fs2-kafka 1.0.3, when I try to use deleteTopic on the AdminApiClient and then createTopic, nondeterministically I will get a failed effect indicating that the topic already exists, i.e. hasn't been deleted by the time deleteTopic passes control to the next effect. Looking at the code, this suggests to me that the machinery for supporting cancelable, which apparently lifts a KafkaFuture to the F context with an Async constraint, is not working correctly (which frankly seems understandable, since KafkaFuture is apparently its own wild-west thing).

paul-snively commented 3 years ago

I have to apologize. I meant fs2-kafka 2.2.0. :-)

vlovgr commented 3 years ago

This happens because deleteTopic might complete before the change has propagated to all brokers, see here. I'm not sure if there's more that can be done except to retry the creation.

bplommer commented 2 years ago

I think we could add something about this to the scaladoc.

aartigao commented 8 months ago

Since KafkaFuture now uses CompletionStage, this should be fixed.

I think it's safe to close this 2 year old issue.