wsky / top-link

embedded duplex multi-channel endpoint and connection management for c#/java/...
6 stars 1 forks source link

using ClientChannelPooledSelector got high load that too many nio threads #79

Closed wsky closed 11 years ago

wsky commented 11 years ago

in RemotingPerfTest

https://github.com/wsky/top-link/blob/master/java/src/test/java/com/taobao/top/link/remoting/RemotingPerf.java

proxy = new DynamicProxy(uri, new ClientChannelPooledSelector(), handler);
//proxy = new DynamicProxy(uri, new ClientChannelSharedSelector(), handler);

if using ClientChannelPooledSelector, low tps and high load that upper to 30+,

while ClientChannelSharedSelector, load only 2+

more concurrent, more io threads

currently, netty client init with following code:

private static ClientBootstrap prepareBootstrap(Logger logger,
            final ChannelPipeline pipeline,
            ChannelHandler handler,
            SslHandler sslHandler,
            int connectTimeoutMillis) {
        ClientBootstrap bootstrap = new ClientBootstrap(new NioClientSocketChannelFactory(
                Executors.newCachedThreadPool(),
                Executors.newCachedThreadPool()));

        bootstrap.setOption("tcpNoDelay", true);
        bootstrap.setOption("reuseAddress", true);
        bootstrap.setOption("connectTimeoutMillis", connectTimeoutMillis);
        bootstrap.setOption("writeBufferHighWaterMark", 10 * 1024 * 1024);

        if (sslHandler != null)
            pipeline.addFirst("ssl", sslHandler);
        if (handler != null)
            pipeline.addLast("handler", handler);

        bootstrap.setPipelineFactory(new ChannelPipelineFactory() {
            @Override
            public ChannelPipeline getPipeline() throws Exception {
                return pipeline;
            }
        });
        return bootstrap;
    }

the threadpool usage so bad for remoting usage, maybe we need sharded threadpool

Executors.newCachedThreadPool()
wsky commented 11 years ago

using fixed pool

Executors.newFixedThreadPool(1)
wsky commented 11 years ago

after init more than 100 netty clientbootstrap, load up to 100+, even no request

more than 1600+ threads for the process

ClientBootstrap bootstrap = new ClientBootstrap(new NioClientSocketChannelFactory(
                Executors.newCachedThreadPool(),
                Executors.newCachedThreadPool()));
wsky commented 11 years ago

shared NioClientSocketChannelFactory can resovle it

static NioClientSocketChannelFactory channelFactory=new NioClientSocketChannelFactory(
        Executors.newCachedThreadPool(),
        Executors.newCachedThreadPool());

but release will also close channelFactory...

bootstrap.releaseExternalResources();
wsky commented 11 years ago

http://stackoverflow.com/questions/7895964/how-does-the-netty-threading-model-work-in-the-case-of-many-client-connections

http://stackoverflow.com/questions/17933798/netty-what-is-the-right-way-to-share-nioclientsocketchannelfactory-among-multip