Shutdown request blocks if connection attempts fail

[dpasek-senacor]

The call to GrpcClient.shutdown() blocks indefinitely if the client has failed to connect to the EventStoreDB.

Flow:

• Create EventStoreDBClient instance.
• Try to connect to a EventStoreDB server which is not available.
• Issue an shutdown requests to the EventStoreDBClient.

Expected behavior:

• Shutdown request finishes.

Observed behavior:

• Shutdown request blocks indefinitely

The reason is that the Shutdown message in the GrpcClient is ignoried during the "drain" phase of the client after shutdown and the connected CompletableFuture is never completed. The Shutdown message is still handled as the client is not yet shutdown when the reconnect attempts are performed.

[dpasek-senacor]

Hier is a test case for the dead lock:

package com.eventstore.dbclient;

import org.junit.jupiter.api.Test;
import testcontainers.module.ESDBTests;
import testcontainers.module.EventStoreDB;

public class ShutdownDeadLockTest extends ESDBTests {

    @Test
    public void testShutdownNotBlockShutdownIfServerIsNotAvailable() throws Throwable {
        EventStoreDB server = getEmptyServer();
        server.stop();
        EventStoreDBClient client = EventStoreDBClient.create(server.getSettings());

        client.shutdown();

    }
}

[dpasek-senacor]

This solution works, but still has the issue that the shutdown request only completes after all reconnection attempts. Currently it is not possible to cancelling the connection attempts by calling shutdown() on the client. The Shutdown message will be queued inside the message loop after the CreateChannel message, which performs the reconnect attempts.

This issue should not be part of this bugticket, but might be an additional improvement on the client. It should be possible to cancel the reconnection attempts with performing a shutdown of the client instance.

[dpasek-senacor]

Works fine now. Issue has been fixed.