Timed out getting config from Elasticache cluster can fail MemcachedClient's constructor instead of falling back to configuration poller

[trongnhanbmt]

We're using Dynamic client mode and the latest version of the client (1.2.2). Occasionally we found some of our application nodes failing to start because the constructor of MemcachedClient threw RuntimeException when our Memcached cluster was under heavy load.

Stacktrace:

Caused by: java.lang.RuntimeException: Exception waiting for config
    at net.spy.memcached.MemcachedClient.getConfig(MemcachedClient.java:1953)
    at net.spy.memcached.MemcachedClient.initializeClientUsingConfigEndPoint(MemcachedClient.java:329)
    at net.spy.memcached.MemcachedClient.<init>(MemcachedClient.java:294)
    at net.spy.memcached.MemcachedClient.<init>(MemcachedClient.java:229)
... 69 common frames omitted
Caused by: java.util.concurrent.ExecutionException: net.spy.memcached.internal.CheckedOperationTimeoutException: Operation timed out. - failing node: /10.202.157.190:11211
    at net.spy.memcached.internal.OperationFuture.get(OperationFuture.java:180)
    at net.spy.memcached.internal.GetConfigFuture.get(GetConfigFuture.java:46)
    at net.spy.memcached.MemcachedClient.getConfig(MemcachedClient.java:1941)
    ... 95 common frames omitted
Caused by: net.spy.memcached.internal.CheckedOperationTimeoutException: Operation timed out. - failing node: /10.202.157.190:11211
    ... 98 common frames omitted

The issue is in this code block inside MemcachedClient:

    public <T> T getConfig(InetSocketAddress addr, ConfigurationType type, Transcoder<T> tc) {
        try {
            return this.asyncGetConfig(addr, type, tc).get(this.operationTimeout, TimeUnit.MILLISECONDS);
        } catch (InterruptedException var6) {
            throw new RuntimeException("Interrupted waiting for config", var6);
        } catch (OperationNotSupportedException var7) {
            throw var7;
        } catch (ExecutionException var8) {
            if (var8.getCause() instanceof OperationException) {
                OperationException exp = (OperationException)var8.getCause();
                if (OperationErrorType.GENERAL.equals(exp.getType())) {
                    throw new OperationNotSupportedException("This version of getConfig command is not supported.");
                }
            }

            throw new RuntimeException("Exception waiting for config", var8);
        } catch (TimeoutException var9) {
            throw new OperationTimeoutException("Timeout waiting for config", var9);
        }
    }

Since an OperationFuture is returned by the asyncGetConfig, sometimes a CheckedOperationTimeoutException wrapped inside a ExecutionException is thrown by OperationFuture.get when a timeout occurs instead of the usual TimeoutException from Future. This exception is then wrapped inside a RuntimeException causing the client ctor to fail immediately.