Closed GoogleCodeExporter closed 9 years ago
后来有没有恢复?
在此之前的异常能不能贴出来。
Original comment by killme2...@gmail.com
on 28 Jul 2011 at 1:38
嗯嗯.
下面就是一个整体错误快, 其他都是循环这个同样的错误,
偶然出个"There is no available connection at this moment".
-----------
10:03:58.102 [Xmemcached-Reactor-13] WARN c.g.c.y.c.i.AbstractController -
Remove a session: 10.10.2.123:11222
10:04:00.104 [Heal-Session-Thread] WARN c.g.c.y.c.i.AbstractController -
Trying to connect to 10.10.2.123:11222 for 1 times
10:04:00.104 [Xmemcached-Reactor-0] WARN c.g.c.y.c.i.AbstractController - Add
a session: 10.10.2.123:11222
10:04:00.109 [Heal-Session-Thread] WARN c.g.c.y.c.i.AbstractController -
Trying to connect to 10.10.2.123:11222 for 1 times
10:04:00.109 [Xmemcached-Reactor-0] WARN c.g.c.y.c.i.AbstractController - Add
a session: 10.10.2.123:11222
10:08:10.955 [Xmemcached-Reactor-15] WARN c.g.c.y.c.i.AbstractController -
Remove a session: 10.10.2.123:11222
10:08:10.961 ["http-bio-80"-exec-16] ERROR *************** -
net.rubyeye.xmemcached.exception.MemcachedException: Session has been closed
net.rubyeye.xmemcached.exception.MemcachedException:
net.rubyeye.xmemcached.exception.MemcachedException: Session has been closed
at net.rubyeye.xmemcached.XMemcachedClient.checkException(XMemcachedClient.java:1813) ~[xmemcached-1.3.2.jar:na]
at net.rubyeye.xmemcached.XMemcachedClient.fetch0(XMemcachedClient.java:572) ~[xmemcached-1.3.2.jar:na]
at net.rubyeye.xmemcached.XMemcachedClient.get0(XMemcachedClient.java:965) ~[xmemcached-1.3.2.jar:na]
at net.rubyeye.xmemcached.XMemcachedClient.get(XMemcachedClient.java:923) ~[xmemcached-1.3.2.jar:na]
at net.rubyeye.xmemcached.XMemcachedClient.get(XMemcachedClient.java:934) ~[xmemcached-1.3.2.jar:na]
at net.rubyeye.xmemcached.XMemcachedClient.get(XMemcachedClient.java:956) ~[xmemcached-1.3.2.jar:na]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:641) [servlet-api.jar:na]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:722) [servlet-api.jar:na]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:304) [catalina.jar:7.0.16]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) [catalina.jar:7.0.16]
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:240) [catalina.jar:7.0.16]
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:164) [catalina.jar:7.0.16]
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:462) [catalina.jar:7.0.16]
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:164) [catalina.jar:7.0.16]
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100) [catalina.jar:7.0.16]
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:563) [catalina.jar:7.0.16]
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) [catalina.jar:7.0.16]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:403) [catalina.jar:7.0.16]
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:301) [tomcat-coyote.jar:7.0.16]
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:162) [tomcat-coyote.jar:7.0.16]
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:140) [tomcat-coyote.jar:7.0.16]
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:309) [tomcat-coyote.jar:7.0.16]
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [na:1.6.0_24]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [na:1.6.0_24]
at java.lang.Thread.run(Thread.java:662) [na:1.6.0_24]
net.rubyeye.xmemcached.exception.MemcachedException: Session has been closed
at net.rubyeye.xmemcached.impl.MemcachedTCPSession.destroy(MemcachedTCPSession.java:115) ~[xmemcached-1.3.2.jar:na]
at net.rubyeye.xmemcached.impl.MemcachedHandler.onSessionClosed(MemcachedHandler.java:136) ~[xmemcached-1.3.2.jar:na]
at com.google.code.yanf4j.core.impl.AbstractSession.onClosed(AbstractSession.java:305) ~[xmemcached-1.3.2.jar:na]
at com.google.code.yanf4j.core.impl.AbstractSession.close(AbstractSession.java:293) ~[xmemcached-1.3.2.jar:na]
at com.google.code.yanf4j.nio.impl.NioTCPSession.readFromBuffer(NioTCPSession.java:216) ~[xmemcached-1.3.2.jar:na]
at com.google.code.yanf4j.nio.impl.AbstractNioSession.onRead(AbstractNioSession.java:198) ~[xmemcached-1.3.2.jar:na]
at com.google.code.yanf4j.nio.impl.AbstractNioSession.onEvent(AbstractNioSession.java:343) ~[xmemcached-1.3.2.jar:na]
at com.google.code.yanf4j.nio.impl.SocketChannelController.dispatchReadEvent(SocketChannelController.java:56) ~[xmemcached-1.3.2.jar:na]
at com.google.code.yanf4j.nio.impl.NioController.onRead(NioController.java:157) ~[xmemcached-1.3.2.jar:na]
at com.google.code.yanf4j.nio.impl.Reactor.dispatchEvent(Reactor.java:294) ~[xmemcached-1.3.2.jar:na]
at com.google.code.yanf4j.nio.impl.Reactor.run(Reactor.java:141) ~[xmemcached-1.3.2.jar:na]
10:08:10.961 [Xmemcached-Reactor-12] ERROR n.r.x.c.b.BinaryStoreCommand -
Invalid arguments.
Original comment by xqing...@gmail.com
on 28 Jul 2011 at 2:17
我是想看这个异常的完整堆栈
08:59:50.661 [Xmemcached-Reactor-7] ERROR n.r.x.c.b.BinaryStoreCommand -
Invalid arguments.
Original comment by killme2...@gmail.com
on 28 Jul 2011 at 2:20
ERROR n.r.x.c.b.BinaryStoreCommand - Invalid arguments.
这个异常就这么一条. 没有堆栈信息.
Original comment by xqing...@gmail.com
on 28 Jul 2011 at 2:43
你在你们的日志里grep下Invalid
arguments,看看有没有其他日志,可能是被jvm优化忽略调堆栈��
�。
Original comment by killme2...@gmail.com
on 28 Jul 2011 at 2:44
关于"Invalid arguments"就只一条, 没有其他的信息, 整个日志200+M都是这个上面日志循环的..
Original comment by xqing...@gmail.com
on 28 Jul 2011 at 2:56
重启了还有这个现象吗?
我查看下代码,这个问题是由于某个暂时还不知道原因,导��
�xmc到memcached的连接被断开。
我建议你在出问题的机器上,用telnet连上你们的memached,执行
get命令,等待一段时间,看看有没有自动断开,排除网络问��
�。
Original comment by killme2...@gmail.com
on 28 Jul 2011 at 3:00
这个只是一直在报错. 但是项目还是在正常运行. 我先试试你给的方案.
Original comment by xqing...@gmail.com
on 28 Jul 2011 at 3:03
我看到的貌似是10.10.2.123:11222 这台有问题。
Original comment by killme2...@gmail.com
on 28 Jul 2011 at 3:05
我就用了这一台memcached...
Original comment by xqing...@gmail.com
on 28 Jul 2011 at 3:16
后续进展?
Original comment by killme2...@gmail.com
on 28 Jul 2011 at 12:29
现在还在报同样的错误. 现在已经有2G的同样错误日志了..
Original comment by xqing...@gmail.com
on 29 Jul 2011 at 3:31
很诡异,不影响使用吗?你们不重启吗?这个异常很频繁吗��
�大概间隔在多少
Original comment by killme2...@gmail.com
on 29 Jul 2011 at 3:41
晕. 刚又看了下, 日志5G了.. 刷屏一样.
There is no available connection at this moment
Original comment by xqing...@gmail.com
on 29 Jul 2011 at 3:50
重启了?
Original comment by killme2...@gmail.com
on 29 Jul 2011 at 9:53
已经快抗不住了.. 一天就这个的日志大约40G. 并发也就在300/s.
connectionPoolSize=2
------------------------------------------
stats
STAT pid 5405
STAT uptime 5865491
STAT time 1312177759
STAT version 1.4.5
STAT pointer_size 64
STAT rusage_user 10158.398688
STAT rusage_system 20704.232478
STAT curr_connections 15
STAT total_connections 57174
STAT connection_structures 359
STAT cmd_get 1380790177
STAT cmd_set 473245495
STAT cmd_flush 30
STAT get_hits 1319491560
STAT get_misses 61298617
STAT delete_misses 0
STAT delete_hits 2
STAT incr_misses 0
STAT incr_hits 0
STAT decr_misses 0
STAT decr_hits 0
STAT cas_misses 0
STAT cas_hits 0
STAT cas_badval 0
STAT auth_cmds 0
STAT auth_errors 0
STAT bytes_read 315299263037
STAT bytes_written 1306721159367
STAT limit_maxbytes 10307502080
STAT accepting_conns 1
STAT listen_disabled_num 0
STAT threads 4
STAT conn_yields 81098
STAT bytes 9121848893
STAT curr_items 26562683
STAT total_items 473245495
STAT evictions 45568465
STAT reclaimed 37497505
END
Original comment by xqing...@gmail.com
on 1 Aug 2011 at 5:51
读取300G,300/S不至于出这样的问题,检查下网络或者是尝试��
�台服务器。
Original comment by springda...@gmail.com
on 4 Aug 2011 at 1:06
请问下,有没有后续消息?
Original comment by killme2...@gmail.com
on 5 Sep 2011 at 1:07
具体原因没找到. 现改用redis了.. 继续关注xmemcached中.
Original comment by xqing...@gmail.com
on 6 Sep 2011 at 8:52
这个问题很奇怪,不过你们已经改用redis了,遗憾。
我最近修正了二进制协议解析的一个bug,可能会引起连接的��
�正常断开,如果你们还用memcached,也许可以尝试下即将发布�
��1.3.4
Original comment by killme2...@gmail.com
on 7 Sep 2011 at 11:30
Original comment by killme2...@gmail.com
on 26 Mar 2012 at 10:35
我用的版本是 xmemcached-1.2.6.2.jar ; Memcached版本是:1.2.0
也出现一堆的
Caused by: java.util.concurrent.TimeoutException: Timed out(3000) waiting for
operation
at net.rubyeye.xmemcached.XMemcachedClient.latchWait(XMemcachedClient.java:2230) ~[XMemcachedClient.class:na]
at net.rubyeye.xmemcached.XMemcachedClient.fetch0(XMemcachedClient.java:513) ~[XMemcachedClient.class:na]
at net.rubyeye.xmemcached.XMemcachedClient.get0(XMemcachedClient.java:879) ~[XMemcachedClient.class:na]
at net.rubyeye.xmemcached.XMemcachedClient.get(XMemcachedClient.java:837) ~[XMemcachedClient.class:na]
at net.rubyeye.xmemcached.XMemcachedClient.get(XMemcachedClient.java:848) ~[XMemcachedClient.class:na]
at net.rubyeye.xmemcached.XMemcachedClient.get(XMemcachedClient.java:870) ~[XMemcachedClient.class:na]
... 74 common frames omitted
同时伴随着:
05-22 08:36:33.360 WARN c.s.t.c.c.c.i.m.XmemcachedCacheNoRuntimeException:49 -
Memcached Fail Exception: net.rubyeye.xmemcached.exception.MemcachedException:
There is no available connection at this moment
java.lang.RuntimeException:
net.rubyeye.xmemcached.exception.MemcachedException: There is no available
connection at this moment
但是我的没有 net.rubyeye.xmemcached.exception.MemcachedException: Session
has been closed
配置的是一个连接,但是有两个Memcached的服务源,不同的数��
�源。事故发生时,连接状态是正常的:
[@zw_83_84 paicha]# grep 10.11.132.56 netstat_84_fail_0522.txt
tcp 0 0 10.10.83.84:6868 10.11.132.56:11212
ESTABLISHED 25386/java
tcp 0 0 10.10.83.84:54908 10.11.132.56:11211
ESTABLISHED 25386/java
Original comment by teneb...@gmail.com
on 22 May 2012 at 8:37
补充: 日平均是 1.7K Request/Second 读操作远远高于写操作。
Original comment by teneb...@gmail.com
on 22 May 2012 at 8:39
[deleted comment]
@ teneb...@gmail.com
我的也出现了这样状况,你们怎么解决的?
Original comment by wyant...@gmail.com
on 21 Apr 2014 at 4:59
Original issue reported on code.google.com by
xqing...@gmail.com
on 28 Jul 2011 at 1:06