Netflix / dynomite

A generic dynamo implementation for different k-v storage engines
Apache License 2.0
4.2k stars 533 forks source link

dynomite server Connection closing issue #625

Closed duraisridhar closed 5 years ago

duraisridhar commented 5 years ago

Hi ,

Connection is closed intermittently between my application and dynomite. Dynomite running in VMware system .

I see below logs in dynomite

[2019-01-09 16:30:45.999] proxy_accept:214 <CONN_PROXY 0x1ae9db0 260 listening on '172.17.247.114:8102'> accepted <CONN_CLIENT 0x21a1060 6406 from '172.17.247.114:5866 '> [2019-01-09 16:31:01.579] core_close:417 close <CONN_CLIENT 0x21a1060 6406 from '172.17.247.114:58664'> on event 00FF eof 1 done 1 rb 57 sb 7 [2019-01-09 16:31:01.579] client_unref_internal_try_put:94 <CONN_CLIENT 0x21a1060 -1 from '172.17.247.114:58664'> unref owner <POOL 0x1ab9370 'dyn_o_mite'>

In client below error is seen. 2019-01-09 16:31:01 - Failed to close connection for host: Host [hostname=172.17.247.114, ipAddress=null, port=8102, rack: localrack, datacenter: localrac, status: Up, hashtag=null] Unexpected end of stream.

lmapi.2019-01-09.18.log:2019-01-09 16:25:03.628 - Error while fetching details from dynomo PoolOfflineException: [host=Host [hostname=UNKNOWN, ipAddress=UNKNOWN, port=0, rack: UNKNOWN, datacenter: UNKNOW, status: Down, hashtag=null], latency=0(0), attempts=0]host pool is offline and no Racks available for fallback

Regards Sridhar

smukil commented 5 years ago

Hi Sridhar,

What HostSupplier are you using in Dyno? https://github.com/Netflix/dyno/wiki/Configuration#hostsupplier

duraisridhar commented 5 years ago

Hi Sailesh, Thanks for your reply. Issue resolved after increasing the MaxConnsPerHost . Earlier default MaxConnsPerHost value 3 is used.

Below host supplier is used

`public HostSupplier getHostSupplier() {

             hostSupplier = new HostSupplier() {

                @Override
                public List<Host> getHosts() {

                    return Collections.singletonList( new Host('localhost', 8102, 'rack', Host.Status.Up));
                }
            };

    return hostSupplier;
}`