The code I have is using Spring Boot 2.5.1 (Spring Cloud Kafka Binders) with Confluent Kafka 6.1.1
For testing, I'm using TestContainers to start a Kafka broker and Schema Registry to run against the test code
For this issue, I've written a minimum code that allows me to reproduce the issue:
The code is using KafkaContainersOrchestration to create the Kafka broker and Schema registry (using test containers) and register the configuration to Spring Boot. KafkaContainersOrchestration provides methods to start and shut down the Kafka container.
As a side note:
Using a clean instance of Kafka is essential for testing perspective.
Shutting down Kafka is essential since we have multiple test classes, each creating its own instance of Kafka, which consumes resources, making the test run slower and eventually break (since Docker is not able to start an additional container)
Assume you have Test2 which is an identical copy of Test1 and you run both of them.
The first test will start creating the streams, consumers and providers.
The following log is from the first test:
It will finish the test and shut down the containers, moving to the next test.
Instead of having a "clean" application context, it seems that all the previous binding entities linger:
The following log is from the second test:
2022-03-03 11:52:28.322 DEBUG [Selector ] [AdminClient clientId=sow-consumer-supplier-update_consumer-0067ee10-5449-4573-9013-b2a95ff42f09-admin] Connection with localhost/127.0.0.1 disconnected
java.net.ConnectException: Connection refused: no further information
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:777)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:219)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:526)
at org.apache.kafka.common.network.Selector.poll(Selector.java:481)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:563)
at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.processRequests(KafkaAdminClient.java:1329)
at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:1260)
at java.base/java.lang.Thread.run(Thread.java:829)
Further, in the same log of the second test:
2022-03-03 11:51:52.323 WARN [NetworkClient ] [AdminClient clientId=sow-consumer-supplier-update_consumer-0067ee10-5449-4573-9013-b2a95ff42f09-admin] Connection to node 1 (localhost/127.0.0.1:49310) could not be established. Broker may not be available.
Port 49310, mentioned in the log, belonged to the previous instance of the Kafka broker which was shut down by the end of Test1.
These error doesn't affect the test outcome, however, they make the log dirty (you will get dozens of these errors printed in the log for the second test, which will get even worse for any consecutive test). Also, they are a potential issue because a "state" of a previous test is leaking into the next, which shouldn't happen, as each test should start with a clean "state".
Hi,
The code I have is using Spring Boot 2.5.1 (Spring Cloud Kafka Binders) with Confluent Kafka 6.1.1 For testing, I'm using TestContainers to start a Kafka broker and Schema Registry to run against the test code For this issue, I've written a minimum code that allows me to reproduce the issue:
The code is using KafkaContainersOrchestration to create the Kafka broker and Schema registry (using test containers) and register the configuration to Spring Boot. KafkaContainersOrchestration provides methods to start and shut down the Kafka container. As a side note:
Assume you have Test2 which is an identical copy of Test1 and you run both of them. The first test will start creating the streams, consumers and providers. The following log is from the first test:
It will finish the test and shut down the containers, moving to the next test. Instead of having a "clean" application context, it seems that all the previous binding entities linger: The following log is from the second test:
Further, in the same log of the second test:
Port 49310, mentioned in the log, belonged to the previous instance of the Kafka broker which was shut down by the end of Test1.
These error doesn't affect the test outcome, however, they make the log dirty (you will get dozens of these errors printed in the log for the second test, which will get even worse for any consecutive test). Also, they are a potential issue because a "state" of a previous test is leaking into the next, which shouldn't happen, as each test should start with a clean "state".