beyondqinghua
commented
7 years ago @Override
public String getString(String key) {
if (StringUtils.isBlank(key)) {
return null;
}
Jedis jedis = null;
try {
jedis = redisCodisFactory.getJedis();
if (jedis == null) {
return null;
}
return jedis.get(getKey(key));
} catch (Throwable ex) {
logger.error("getString error.key=" + key, ex);
} finally {
returnResource(jedis);
}
return null;
}
@Override
public void setString(String key, String value) {
if (StringUtils.isBlank(key) || value == null) {
return;
}
Jedis jedis = null;
try {
jedis = redisCodisFactory.getJedis();
if (jedis == null) {
return;
}
jedis.set(getKey(key), value);
} catch (Throwable ex) {
logger.error("setString error.key=" + key, ex);
} finally {
returnResource(jedis);
}
}
private void returnResource(Jedis jedis) {
try {
if (jedis == null) {
return;
}
jedis.close();
} catch (Throwable e) {
logger.error("returnResource error.", e);
}
}Jedis code: @Override public void close() { if (dataSource != null) { if (client.isBroken()) { this.dataSource.returnBrokenResource(this); } else { this.dataSource.returnResource(this); } } else { client.close(); } }
Apache9
spinlock
测试跑100W连续多线程塞数据都正常，但是停下放一段时间后，其他小批量的后台任务反而时不时报错： 2016-12-15 20:11:33.439 [Thread-990017] ERROR c.k.f.s.w.redis.codis.test.task.TestCodisRunnable - 990000-测试codis使用时间:8 2016-12-15 20:11:33.784 [Thread-991017] ERROR c.k.f.s.w.redis.codis.test.task.TestCodisRunnable - 991000-测试codis使用时间:8 2016-12-15 20:11:34.076 [Thread-992017] ERROR c.k.f.s.w.redis.codis.test.task.TestCodisRunnable - 992000-测试codis使用时间:3 2016-12-15 20:11:34.401 [Thread-993017] ERROR c.k.f.s.w.redis.codis.test.task.TestCodisRunnable - 993000-测试codis使用时间:7 2016-12-15 20:11:34.724 [Thread-994017] ERROR c.k.f.s.w.redis.codis.test.task.TestCodisRunnable - 994000-测试codis使用时间:9 2016-12-15 20:11:34.972 [Thread-995017] ERROR c.k.f.s.w.redis.codis.test.task.TestCodisRunnable - 995000-测试codis使用时间:13 2016-12-15 20:11:35.363 [Thread-996017] ERROR c.k.f.s.w.redis.codis.test.task.TestCodisRunnable - 996000-测试codis使用时间:5 2016-12-15 20:11:35.657 [Thread-997017] ERROR c.k.f.s.w.redis.codis.test.task.TestCodisRunnable - 997000-测试codis使用时间:8 2016-12-15 20:11:35.943 [Thread-998017] ERROR c.k.f.s.w.redis.codis.test.task.TestCodisRunnable - 998000-测试codis使用时间:2 2016-12-15 20:11:36.210 [Thread-999017] ERROR c.k.f.s.w.redis.codis.test.task.TestCodisRunnable - 999000-测试codis使用时间:1 2016-12-15 20:52:52.489 [localhost-startStop-1-SendThread(10.201.5.16:2180)] INFO org.apache.zookeeper.ClientCnxn - Client session timed out, have not heard from server in 10004ms for sessionid 0x158fd66fd860020, closing socket connection and attempting reconnect 2016-12-15 20:52:52.489 [localhost-startStop-1-SendThread(10.201.5.16:2180)] INFO org.apache.zookeeper.ClientCnxn - Client session timed out, have not heard from server in 13510ms for sessionid 0x158fd66fd86001f, closing socket connection and attempting reconnect 2016-12-15 20:52:53.988 [localhost-startStop-1-EventThread] INFO o.a.curator.framework.state.ConnectionStateManager - State change: SUSPENDED 2016-12-15 20:52:53.990 [localhost-startStop-1-EventThread] INFO o.a.curator.framework.state.ConnectionStateManager - State change: SUSPENDED 2016-12-15 20:52:54.407 [localhost-startStop-1-SendThread(10.201.5.17:2180)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server 10.201.5.17/10.201.5.17:2180. Will not attempt to authenticate using SASL (unknown error) 2016-12-15 20:52:54.515 [localhost-startStop-1-SendThread(10.201.5.17:2180)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to 10.201.5.17/10.201.5.17:2180, initiating session 2016-12-15 20:52:54.923 [localhost-startStop-1-SendThread(10.201.5.17:2180)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server 10.201.5.17/10.201.5.17:2180. Will not attempt to authenticate using SASL (unknown error) 2016-12-15 20:52:54.924 [localhost-startStop-1-SendThread(10.201.5.17:2180)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to 10.201.5.17/10.201.5.17:2180, initiating session 2016-12-15 20:52:55.625 [localhost-startStop-1-SendThread(10.201.5.17:2180)] INFO org.apache.zookeeper.ClientCnxn - Unable to reconnect to ZooKeeper service, session 0x158fd66fd860020 has expired, closing socket connection 2016-12-15 20:52:55.625 [localhost-startStop-1-SendThread(10.201.5.17:2180)] INFO org.apache.zookeeper.ClientCnxn - Unable to reconnect to ZooKeeper service, session 0x158fd66fd86001f has expired, closing socket connection 2016-12-15 20:52:55.742 [localhost-startStop-1-EventThread] WARN org.apache.curator.ConnectionState - Session expired event received 2016-12-15 20:52:55.823 [Curator-PathChildrenCache-0] INFO io.codis.jodis.RoundRobinJedisPool - Receive child event: type=CONNECTION_SUSPENDED, no data 2016-12-15 20:52:55.823 [Curator-PathChildrenCache-1] INFO io.codis.jodis.RoundRobinJedisPool - Receive child event: type=CONNECTION_SUSPENDED, no data 2016-12-15 20:52:55.824 [localhost-startStop-1-EventThread] INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=10.201.5.16:2180,10.201.5.15:2180,10.201.5.17:2180 sessionTimeout=20000 watcher=org.apache.curator.ConnectionState@7ebe36cc 2016-12-15 20:52:55.825 [localhost-startStop-1-EventThread] WARN org.apache.curator.ConnectionState - Session expired event received 2016-12-15 20:52:55.825 [localhost-startStop-1-EventThread] INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=10.201.5.16:2180,10.201.5.15:2180,10.201.5.17:2180 sessionTimeout=15000 watcher=org.apache.curator.ConnectionState@500764ed 2016-12-15 20:52:55.826 [localhost-startStop-1-EventThread] INFO o.a.curator.framework.state.ConnectionStateManager - State change: LOST 2016-12-15 20:52:55.826 [Curator-PathChildrenCache-0] INFO io.codis.jodis.RoundRobinJedisPool - Receive child event: type=CONNECTION_LOST, no data 2016-12-15 20:52:55.826 [localhost-startStop-1-EventThread] INFO org.apache.zookeeper.ClientCnxn - EventThread shut down 2016-12-15 20:52:55.827 [localhost-startStop-1-SendThread(10.201.5.15:2180)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server 10.201.5.15/10.201.5.15:2180. Will not attempt to authenticate using SASL (unknown error) 2016-12-15 20:52:55.827 [localhost-startStop-1-SendThread(10.201.5.17:2180)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server 10.201.5.17/10.201.5.17:2180. Will not attempt to authenticate using SASL (unknown error) 2016-12-15 20:52:55.828 [localhost-startStop-1-EventThread] INFO o.a.curator.framework.state.ConnectionStateManager - State change: LOST 2016-12-15 20:52:55.828 [localhost-startStop-1-EventThread] INFO org.apache.zookeeper.ClientCnxn - EventThread shut down 2016-12-15 20:52:55.829 [localhost-startStop-1-SendThread(10.201.5.15:2180)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to 10.201.5.15/10.201.5.15:2180, initiating session 2016-12-15 20:52:55.829 [Curator-PathChildrenCache-1] INFO io.codis.jodis.RoundRobinJedisPool - Receive child event: type=CONNECTION_LOST, no data 2016-12-15 20:52:55.829 [localhost-startStop-1-SendThread(10.201.5.17:2180)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to 10.201.5.17/10.201.5.17:2180, initiating session 2016-12-15 20:52:55.832 [localhost-startStop-1-SendThread(10.201.5.17:2180)] INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on server 10.201.5.17/10.201.5.17:2180, sessionid = 0x258faae399a0029, negotiated timeout = 15000 2016-12-15 20:52:55.832 [localhost-startStop-1-EventThread] INFO o.a.curator.framework.state.ConnectionStateManager - State change: RECONNECTED 2016-12-15 20:52:55.833 [Curator-PathChildrenCache-1] INFO io.codis.jodis.RoundRobinJedisPool - Receive child event: type=CONNECTION_RECONNECTED, no data 2016-12-15 20:52:55.834 [localhost-startStop-1-SendThread(10.201.5.15:2180)] INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on server 10.201.5.15/10.201.5.15:2180, sessionid = 0x58faae39ef0026, negotiated timeout = 20000 2016-12-15 20:52:55.834 [localhost-startStop-1-EventThread] INFO o.a.curator.framework.state.ConnectionStateManager - State change: RECONNECTED 2016-12-15 20:52:55.835 [Curator-PathChildrenCache-0] INFO io.codis.jodis.RoundRobinJedisPool - Receive child event: type=CONNECTION_RECONNECTED, no data 2016-12-15 20:53:38.374 [Timer-6] ERROR c.k.finance.redis.service.RedisCodisServiceImpl - leftPopList 出错[key=MQ:600000-system_rank_msg-500000-test] redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: Read timed out at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:201) ~[jedis-2.8.0.jar:na] at redis.clients.util.RedisInputStream.readByte(RedisInputStream.java:40) ~[jedis-2.8.0.jar:na] at redis.clients.jedis.Protocol.process(Protocol.java:141) ~[jedis-2.8.0.jar:na] at redis.clients.jedis.Protocol.read(Protocol.java:205) ~[jedis-2.8.0.jar:na] at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:297) ~[jedis-2.8.0.jar:na] at redis.clients.jedis.Connection.getBinaryBulkReply(Connection.java:216) ~[jedis-2.8.0.jar:na] at redis.clients.jedis.BinaryJedis.lpop(BinaryJedis.java:1153) ~[jedis-2.8.0.jar:na] at com.kingdee.finance.redis.service.RedisCodisServiceImpl.leftPopList(RedisCodisServiceImpl.java:986) ~[finance-lib-redis-1.0.4-SNAPSHOT.jar:1.0.4-SNAPSHOT] at com.kingdee.finance.jms.redismq.queue.manage.MessageTransactionManage.consumeMessage(MessageTransactionManage.java:63) [finance-lib-redisjms-1.0.4-SNAPSHOT.jar:1.0.4-SNAPSHOT] at com.kingdee.finance.jms.redismq.queue.consumer.subscriber.RedisMQQueueSubscriber.subscriber(RedisMQQueueSubscriber.java:56) [finance-lib-redisjms-1.0.4-SNAPSHOT.jar:1.0.4-SNAPSHOT] at com.kingdee.finance.jms.redismq.queue.consumer.subscriber.QueueSubscriberTimerTask.run(QueueSubscriberTimerTask.java:22) [finance-lib-redisjms-1.0.4-SNAPSHOT.jar:1.0.4-SNAPSHOT] at java.util.TimerThread.mainLoop(Timer.java:555) [na:1.7.0_79] at java.util.TimerThread.run(Timer.java:505) [na:1.7.0_79] Caused by: java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) ~[na:1.7.0_79] at java.net.SocketInputStream.read(SocketInputStream.java:152) ~[na:1.7.0_79] at java.net.SocketInputStream.read(SocketInputStream.java:122) ~[na:1.7.0_79] at java.net.SocketInputStream.read(SocketInputStream.java:108) ~[na:1.7.0_79] at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:195) ~[jedis-2.8.0.jar:na] ... 12 common frames omitted 2016-12-15 20:56:11.162 [Timer-6] ERROR c.k.finance.redis.service.RedisCodisServiceImpl - leftPopList 出错[key=MQ:600000-system_rank_msg-500000-test] redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: Read timed out at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:201) ~[jedis-2.8.0.jar:na] at redis.clients.util.RedisInputStream.readByte(RedisInputStream.java:40) ~[jedis-2.8.0.jar:na] at redis.clients.jedis.Protocol.process(Protocol.java:141) ~[jedis-2.8.0.jar:na] at redis.clients.jedis.Protocol.read(Protocol.java:205) ~[jedis-2.8.0.jar:na] at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:297) ~[jedis-2.8.0.jar:na] at redis.clients.jedis.Connection.getBinaryBulkReply(Connection.java:216) ~[jedis-2.8.0.jar:na] at redis.clients.jedis.BinaryJedis.lpop(BinaryJedis.java:1153) ~[jedis-2.8.0.jar:na] at com.kingdee.finance.redis.service.RedisCodisServiceImpl.leftPopList(RedisCodisServiceImpl.java:986) ~[finance-lib-redis-1.0.4-SNAPSHOT.jar:1.0.4-SNAPSHOT] at com.kingdee.finance.jms.redismq.queue.manage.MessageTransactionManage.consumeMessage(MessageTransactionManage.java:63) [finance-lib-redisjms-1.0.4-SNAPSHOT.jar:1.0.4-SNAPSHOT] at com.kingdee.finance.jms.redismq.queue.consumer.subscriber.RedisMQQueueSubscriber.subscriber(RedisMQQueueSubscriber.java:56) [finance-lib-redisjms-1.0.4-SNAPSHOT.jar:1.0.4-SNAPSHOT] at com.kingdee.finance.jms.redismq.queue.consumer.subscriber.QueueSubscriberTimerTask.run(QueueSubscriberTimerTask.java:22) [finance-lib-redisjms-1.0.4-SNAPSHOT.jar:1.0.4-SNAPSHOT] at java.util.TimerThread.mainLoop(Timer.java:555) [na:1.7.0_79] at java.util.TimerThread.run(Timer.java:505) [na:1.7.0_79] Caused by: java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) ~[na:1.7.0_79] at java.net.SocketInputStream.read(SocketInputStream.java:152) ~[na:1.7.0_79] at java.net.SocketInputStream.read(SocketInputStream.java:122) ~[na:1.7.0_79] at java.net.SocketInputStream.read(SocketInputStream.java:108) ~[na:1.7.0_79] at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:195) ~[jedis-2.8.0.jar:na] ... 12 common frames omitted
代码如下： /**
@return */ private JedisResourcePool buildJedisResourcePool(RedisCodisSettingProperties settingProperties,CuratorFramework curatorFramework){
}
/**
@return */ private CuratorFramework buildCuratorFramework(RedisCodisSettingProperties settingProperties) {
}
/**
@return */ private synchronized JedisPoolConfig buildJedisPoolConfig() {
}
参数： session_tms:20000 sleepms_retry:5000
pool_max_total:2000 pool_max_idle:100 pool_min_idle:50 pool_timeout_ms:6000