mondain / red5

Automatically exported from code.google.com/p/red5
3 stars 8 forks source link

Running at 100% CPU due to using org.red5.io.amf.Output.getField() #525

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago
Hi guys.
I am WebApplicationServer egineer. 
I'm not good at writing , so plz understand.
customer complained was cpu usage .
it was almost go up to 100%

so i check all thread dump. all worker thread was running. ( i will attached 
thread dump)
here thread part of full thread dump

     "webtob_server1-hth0-30 [pomdapl2_server1-173]" prio=10 tid=0x00007fdc3c0ed800 nid=0x542e runnable [0x00007fdb2f800000]
        java.lang.Thread.State: RUNNABLE
        at java.util.HashMap.getEntry(HashMap.java:347)
        at red5.containsKey(HashMap.java:335)
        at org.red5.io.amf.Output.getField(Output.java:400)
        at org.red5.io.amf3.Output.writeObject(Output.java:474)
        at org.red5.io.object.Serializer.writeObjectType(Serializer.java:296)
        at org.red5.io.object.Serializer.writeComplex(Serializer.java:140)
        at org.red5.io.object.Serializer.serialize(Serializer.java:86)
        at org.red5.io.object.Serializer.serialize(Serializer.java:63)
        at org.red5.io.amf3.Output.writeArray(Output.java:247)
        at org.red5.io.object.Serializer.writeList(Serializer.java:178)
        at org.red5.io.object.Serializer.writeListType(Serializer.java:160)
        at org.red5.io.object.Serializer.writeComplex(Serializer.java:132)
        at org.red5.io.object.Serializer.serialize(Serializer.java:86)
        at org.red5.io.object.Serializer.serialize(Serializer.java:63)
        at org.red5.io.amf3.Output.writeMap(Output.java:337)
        at org.red5.io.object.Serializer.writeObjectType(Serializer.java:292)
        at org.red5.io.object.Serializer.writeComplex(Serializer.java:140)
        at org.red5.io.object.Serializer.serialize(Serializer.java:86)
        at org.red5.io.amf3.Output.writeObject(Output.java:483)
        at org.red5.io.object.Serializer.writeObjectType(Serializer.java:296)
        at org.red5.io.object.Serializer.writeComplex(Serializer.java:140)
        at org.red5.io.object.Serializer.serialize(Serializer.java:86)
        at org.red5.io.amf3.Output.writeObject(Output.java:483)
        at org.red5.io.object.Serializer.writeObjectType(Serializer.java:296)
        at org.red5.io.object.Serializer.writeComplex(Serializer.java:140)
        at org.red5.io.object.Serializer.serialize(Serializer.java:86)
        at org.red5.io.object.Serializer.serialize(Serializer.java:63)
        at org.red5.server.net.remoting.codec.RemotingProtocolEncoder.encode(RemotingProtocolEncoder.java:136)
        at org.red5.server.net.servlet.AMFGatewayServlet.sendResponse(AMFGatewayServlet.java:278)
        at org.red5.server.net.servlet.AMFGatewayServlet.serviceAMF(AMFGatewayServlet.java:190)
        at org.red5.server.net.servlet.AMFGatewayServlet.service(AMFGatewayServlet.java:119)
        at com.wemb.tobit.TobitGatewayServlet.service(TobitGatewayServlet.java:31)
      .....
        at jeus.util.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:1220)
        at jeus.servlet.engine.WebThreadPoolExecutor$WebRequestWorker.run(WebThreadPoolExecutor.java:332)
        at java.lang.Thread.run(Thread.java:662)

i dumped thread 4time (interval 3~5 second).
all thread is hanging at same stack.

i found that The HashMap.getEntry() method can cause the CPU to spin when 
accessed/modified concurrently.
(http://knowledgebase.progress.com/articles/Article/000035316)

so my options is that some source is motified 
(maybe in org.red5.io.amf.Output.getField() fieldMap shoud be ConcurrentHashMap 
not HashMap

what do you think about it?

Original issue reported on code.google.com by cradl...@gmail.com on 12 Oct 2014 at 5:05

Attachments:

GoogleCodeExporter commented 9 years ago
Ill take a look at it

Original comment by mondain on 14 Oct 2014 at 8:16

GoogleCodeExporter commented 9 years ago

Original comment by mondain on 9 Nov 2014 at 3:57